[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:59:52 dist-test-slave-0kls krb5kdc[13755](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:59:52 dist-test-slave-0kls krb5kdc[13755](info): set up 2 sockets
Aug 11 18:59:52 dist-test-slave-0kls krb5kdc[13755](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260811 18:59:53.621858 13748 mini_kdc.cc:121] Time spent starting KDC: real 1.083s user 0.007s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/krb5kdc/test-user.keytab.
Aug 11 18:59:53 dist-test-slave-0kls krb5kdc[13755](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474793, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:59:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:59:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.109.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.109.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.109.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab.
Entry for principal kudu/127.13.109.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab.
I20260811 18:59:53.757928 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.62:34281
--webserver_interface=127.13.109.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab
--principal=kudu/127.13.109.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:41623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.109.62:34281
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:59:53.878554 13770 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:59:53.878908 13770 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:59:53.878994 13770 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:59:53.883091 13770 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:59:53.883185 13770 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:59:53.883224 13770 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:59:53.883286 13770 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:59:53.883324 13770 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:59:53.888836 13770 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:41623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.109.62:34281
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab
--principal=kudu/127.13.109.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.62:34281
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.109.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13770
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:59:53.890332 13770 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:59:53.891486 13770 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:59:53.901209 13776 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:53.901209 13775 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:53.901278 13778 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:59:53.901312 13770 server_base.cc:1034] running on GCE node
I20260811 18:59:53.902164 13770 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:59:53.903406 13770 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:59:53.904614 13770 hybrid_clock.cc:648] HybridClock initialized: now 1786474793904601 us; error 33 us; skew 500 ppm
Aug 11 18:59:53 dist-test-slave-0kls krb5kdc[13755](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474793, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:59:53.907477 13770 init.cc:377] Logged in from keytab as kudu/127.13.109.62@KRBTEST.COM (short username kudu)
I20260811 18:59:53.908428 13770 webserver.cc:460] Webserver started at http://127.13.109.62:34919/ using document root <none> and password file <none>
I20260811 18:59:53.909029 13770 fs_manager.cc:359] Metadata directory not provided
I20260811 18:59:53.909121 13770 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:59:53.909443 13770 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:59:53.911795 13770 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/minicluster-data/master-0/data/instance:
uuid: "cc67d1c11ba2485494025631a59aa9b5"
format_stamp: "Formatted at 2026-08-11 18:59:53 on dist-test-slave-0kls"
server_key: "1ee676c88b20b3dfd5514a9e6bacca36"
server_key_iv: "fb7b50035de4a8ea82a027cf17d2836b"
server_key_version: "encryptionkey@0"
I20260811 18:59:53.912432 13770 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/minicluster-data/master-0/wal/instance:
uuid: "cc67d1c11ba2485494025631a59aa9b5"
format_stamp: "Formatted at 2026-08-11 18:59:53 on dist-test-slave-0kls"
server_key: "1ee676c88b20b3dfd5514a9e6bacca36"
server_key_iv: "fb7b50035de4a8ea82a027cf17d2836b"
server_key_version: "encryptionkey@0"
I20260811 18:59:53.916652 13770 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 18:59:53.919628 13787 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:53.921032 13770 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 18:59:53.921187 13770 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/minicluster-data/master-0/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/minicluster-data/master-0/wal
uuid: "cc67d1c11ba2485494025631a59aa9b5"
format_stamp: "Formatted at 2026-08-11 18:59:53 on dist-test-slave-0kls"
server_key: "1ee676c88b20b3dfd5514a9e6bacca36"
server_key_iv: "fb7b50035de4a8ea82a027cf17d2836b"
server_key_version: "encryptionkey@0"
I20260811 18:59:53.921320 13770 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:59:53.972729 13770 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:59:53.976159 13770 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:59:53.976398 13770 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:59:53.984813 13770 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.62:34281
I20260811 18:59:53.984848 13841 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.62:34281 every 8 connection(s)
I20260811 18:59:53.985896 13770 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
I20260811 18:59:53.987106 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 13770
I20260811 18:59:53.987288 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/minicluster-data/master-0/wal/instance
I20260811 18:59:53.989025 13748 external_mini_cluster.cc:1428] Setting key 34cc5ce2a10a99f5ff7b60b44186e01c
I20260811 18:59:53.989578 13842 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:59:53.997030 13842 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc67d1c11ba2485494025631a59aa9b5: Bootstrap starting.
I20260811 18:59:54.000048 13842 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cc67d1c11ba2485494025631a59aa9b5: Neither blocks nor log segments found. Creating new log.
I20260811 18:59:54.001106 13842 log.cc:824] T 00000000000000000000000000000000 P cc67d1c11ba2485494025631a59aa9b5: Log is configured to *not* fsync() on all Append() calls
I20260811 18:59:54.003662 13842 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc67d1c11ba2485494025631a59aa9b5: No bootstrap required, opened a new log
I20260811 18:59:54.008888 13842 raft_consensus.cc:348] T 00000000000000000000000000000000 P cc67d1c11ba2485494025631a59aa9b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc67d1c11ba2485494025631a59aa9b5" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 34281 } }
I20260811 18:59:54.009177 13842 raft_consensus.cc:374] T 00000000000000000000000000000000 P cc67d1c11ba2485494025631a59aa9b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:59:54.009234 13842 raft_consensus.cc:729] T 00000000000000000000000000000000 P cc67d1c11ba2485494025631a59aa9b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc67d1c11ba2485494025631a59aa9b5, State: Initialized, Role: FOLLOWER
I20260811 18:59:54.009943 13842 consensus_queue.cc:260] T 00000000000000000000000000000000 P cc67d1c11ba2485494025631a59aa9b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc67d1c11ba2485494025631a59aa9b5" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 34281 } }
I20260811 18:59:54.010138 13842 raft_consensus.cc:388] T 00000000000000000000000000000000 P cc67d1c11ba2485494025631a59aa9b5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:59:54.010216 13842 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc67d1c11ba2485494025631a59aa9b5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:59:54.010326 13842 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cc67d1c11ba2485494025631a59aa9b5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:59:54.011742 13842 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc67d1c11ba2485494025631a59aa9b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc67d1c11ba2485494025631a59aa9b5" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 34281 } }
I20260811 18:59:54.012156 13842 leader_election.cc:302] T 00000000000000000000000000000000 P cc67d1c11ba2485494025631a59aa9b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cc67d1c11ba2485494025631a59aa9b5; no voters:
I20260811 18:59:54.012514 13842 leader_election.cc:288] T 00000000000000000000000000000000 P cc67d1c11ba2485494025631a59aa9b5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:59:54.012660 13847 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cc67d1c11ba2485494025631a59aa9b5 [term 1 FOLLOWER]: Leader election won for term 1
Aug 11 18:59:54 dist-test-slave-0kls krb5kdc[13755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474793, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
I20260811 18:59:54.012909 13847 raft_consensus.cc:686] T 00000000000000000000000000000000 P cc67d1c11ba2485494025631a59aa9b5 [term 1 LEADER]: Becoming Leader. State: Replica: cc67d1c11ba2485494025631a59aa9b5, State: Running, Role: LEADER
I20260811 18:59:54.013283 13847 consensus_queue.cc:237] T 00000000000000000000000000000000 P cc67d1c11ba2485494025631a59aa9b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc67d1c11ba2485494025631a59aa9b5" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 34281 } }
I20260811 18:59:54.013988 13842 sys_catalog.cc:564] T 00000000000000000000000000000000 P cc67d1c11ba2485494025631a59aa9b5 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:59:54.015837 13848 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc67d1c11ba2485494025631a59aa9b5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cc67d1c11ba2485494025631a59aa9b5. Latest consensus state: current_term: 1 leader_uuid: "cc67d1c11ba2485494025631a59aa9b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc67d1c11ba2485494025631a59aa9b5" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 34281 } } }
I20260811 18:59:54.016007 13848 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc67d1c11ba2485494025631a59aa9b5 [sys.catalog]: This master's current role is: LEADER
I20260811 18:59:54.016511 13858 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:59:54.017163 13849 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc67d1c11ba2485494025631a59aa9b5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cc67d1c11ba2485494025631a59aa9b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc67d1c11ba2485494025631a59aa9b5" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 34281 } } }
I20260811 18:59:54.017488 13849 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc67d1c11ba2485494025631a59aa9b5 [sys.catalog]: This master's current role is: LEADER
I20260811 18:59:54.020149 13858 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:59:54.022014 13844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:53.993163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53456 (local address 127.13.109.62:34281)
0811 18:59:53.993698 (+ 535us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:53.993719 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:53.994787 (+ 1068us) server_negotiation.cc:407] Connection header received
0811 18:59:53.999164 (+ 4377us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:53.999181 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:53.999533 (+ 352us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:53.999844 (+ 311us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:54.003303 (+ 3459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:54.004837 (+ 1534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:54.006847 (+ 2010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:54.007777 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:54.007980 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:54.013853 (+ 5873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:54.013861 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:54.013879 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:59:54.013919 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:54.017446 (+ 3527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:54.018272 (+ 826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:54.018279 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:54.018285 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:54.018436 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:54.018936 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:54.018939 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:54.018941 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:54.019526 (+ 585us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:54.019768 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:54.020600 (+ 832us) server_negotiation.cc:299] Negotiation successful
0811 18:59:54.020939 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":139,"threads_started":1}
I20260811 18:59:54.027323 13858 catalog_manager.cc:1347] Generated new cluster ID: 44bb0cf7d5f04a999f7fc808cadc11d3
I20260811 18:59:54.027410 13858 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:59:54.039594 13858 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:59:54.041018 13858 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:59:54.047192 13858 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cc67d1c11ba2485494025631a59aa9b5: Generated new TSK 0
I20260811 18:59:54.047983 13858 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.109.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.109.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.109.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.1.keytab.
Entry for principal kudu/127.13.109.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.1.keytab.
I20260811 18:59:54.090919 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.1:0
--local_ip_for_outbound_sockets=127.13.109.1
--webserver_interface=127.13.109.1
--webserver_port=0
--tserver_master_addrs=127.13.109.62:34281
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.1.keytab
--principal=kudu/127.13.109.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:41623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:59:54.211501 13869 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:59:54.211797 13869 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:59:54.211843 13869 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:59:54.216235 13869 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:59:54.216305 13869 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:59:54.216365 13869 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.1
I20260811 18:59:54.221887 13869 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:41623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.1.keytab
--principal=kudu/127.13.109.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.109.1
--webserver_port=0
--tserver_master_addrs=127.13.109.62:34281
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13869
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.1
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:59:54.223007 13869 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:59:54.224026 13869 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:59:54.234622 13877 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:54.234745 13874 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:54.234687 13875 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:59:54.234922 13869 server_base.cc:1034] running on GCE node
I20260811 18:59:54.235535 13869 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:59:54.236286 13869 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:59:54.237499 13869 hybrid_clock.cc:648] HybridClock initialized: now 1786474794237473 us; error 41 us; skew 500 ppm
Aug 11 18:59:54 dist-test-slave-0kls krb5kdc[13755](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474794, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:59:54.240622 13869 init.cc:377] Logged in from keytab as kudu/127.13.109.1@KRBTEST.COM (short username kudu)
I20260811 18:59:54.241551 13869 webserver.cc:460] Webserver started at http://127.13.109.1:39309/ using document root <none> and password file <none>
I20260811 18:59:54.242218 13869 fs_manager.cc:359] Metadata directory not provided
I20260811 18:59:54.242285 13869 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:59:54.242462 13869 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:59:54.244560 13869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/minicluster-data/ts-0/data/instance:
uuid: "e559642c61db49f0929ababd8450a65d"
format_stamp: "Formatted at 2026-08-11 18:59:54 on dist-test-slave-0kls"
server_key: "fce9438141f98af3fee36ce5e63676d9"
server_key_iv: "fc0f1e5cce79a4ad5f5caa211e1268d7"
server_key_version: "encryptionkey@0"
I20260811 18:59:54.245105 13869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/minicluster-data/ts-0/wal/instance:
uuid: "e559642c61db49f0929ababd8450a65d"
format_stamp: "Formatted at 2026-08-11 18:59:54 on dist-test-slave-0kls"
server_key: "fce9438141f98af3fee36ce5e63676d9"
server_key_iv: "fc0f1e5cce79a4ad5f5caa211e1268d7"
server_key_version: "encryptionkey@0"
I20260811 18:59:54.249151 13869 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 18:59:54.251641 13884 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:54.252864 13869 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 18:59:54.252959 13869 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/minicluster-data/ts-0/wal
uuid: "e559642c61db49f0929ababd8450a65d"
format_stamp: "Formatted at 2026-08-11 18:59:54 on dist-test-slave-0kls"
server_key: "fce9438141f98af3fee36ce5e63676d9"
server_key_iv: "fc0f1e5cce79a4ad5f5caa211e1268d7"
server_key_version: "encryptionkey@0"
I20260811 18:59:54.253044 13869 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:59:54.280895 13869 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:59:54.283824 13869 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:59:54.283987 13869 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:59:54.284570 13869 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:59:54.285418 13869 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:59:54.285466 13869 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:54.285509 13869 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:59:54.285524 13869 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:54.295536 13869 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.1:43965
I20260811 18:59:54.295571 14009 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.1:43965 every 8 connection(s)
I20260811 18:59:54.296459 13869 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
I20260811 18:59:54.299289 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 13869
I20260811 18:59:54.299412 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/minicluster-data/ts-0/wal/instance
I20260811 18:59:54.299744 13748 external_mini_cluster.cc:1428] Setting key d6c369ab6bd3a0d9d4c946cfcc1c5cf3
Aug 11 18:59:54 dist-test-slave-0kls krb5kdc[13755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474794, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.1@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.109.2@KRBTEST.COM; defaulting to no policy
I20260811 18:59:54.311874 13844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:54.298503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.1:45705 (local address 127.13.109.62:34281)
0811 18:59:54.298644 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:54.298647 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:54.299629 (+ 982us) server_negotiation.cc:407] Connection header received
0811 18:59:54.301375 (+ 1746us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:54.301377 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:54.301445 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:54.301531 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:54.302936 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:54.303724 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:54.304980 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:54.305218 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:54.305306 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:54.308207 (+ 2901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:54.308210 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:54.308213 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:59:54.308236 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:54.309673 (+ 1437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:54.310280 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:54.310284 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:54.310286 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:54.310373 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:54.310752 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:54.310756 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:54.310757 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:54.310945 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:54.311065 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:54.311568 (+ 503us) server_negotiation.cc:299] Negotiation successful
0811 18:59:54.311696 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 18:59:54.312687 14012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:54.298928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:34281 (local address 127.13.109.1:45705)
0811 18:59:54.299443 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:54.299501 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 18:59:54.301114 (+ 1613us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:59:54.301709 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:59:54.301716 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:59:54.302264 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:59:54.302778 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:54.302793 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:54.303857 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:54.303861 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:54.304838 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:54.304854 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:54.305384 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:54.305390 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:54.305631 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:54.306288 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:59:54.306313 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:59:54.308031 (+ 1718us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:59:54.309854 (+ 1823us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:59:54.309860 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:59:54.309875 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:59:54.310171 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:59:54.310470 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:59:54.310474 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:59:54.310476 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:59:54.310641 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:59:54.311058 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:59:54.311064 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:59:54.311324 (+ 260us) client_negotiation.cc:769] Sending connection context
0811 18:59:54.311510 (+ 186us) client_negotiation.cc:240] Negotiation successful
0811 18:59:54.311843 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":161,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.109.2@KRBTEST.COM" created.
I20260811 18:59:54.314316 14010 heartbeater.cc:347] Connected to a master server at 127.13.109.62:34281
I20260811 18:59:54.314591 14010 heartbeater.cc:464] Registering TS with master...
I20260811 18:59:54.315214 14010 heartbeater.cc:511] Master 127.13.109.62:34281 requested a full tablet report, sending...
I20260811 18:59:54.317183 13805 ts_manager.cc:194] Registered new tserver with Master: e559642c61db49f0929ababd8450a65d (127.13.109.1:43965)
I20260811 18:59:54.319003 13805 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.109.1@KRBTEST.COM'} at 127.13.109.1:45705
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.109.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.2.keytab.
Entry for principal kudu/127.13.109.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.2.keytab.
I20260811 18:59:54.332645 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.2:0
--local_ip_for_outbound_sockets=127.13.109.2
--webserver_interface=127.13.109.2
--webserver_port=0
--tserver_master_addrs=127.13.109.62:34281
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.2.keytab
--principal=kudu/127.13.109.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:41623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:59:54.451293 14015 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:59:54.451582 14015 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:59:54.451630 14015 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:59:54.455812 14015 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:59:54.455879 14015 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:59:54.455979 14015 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.2
I20260811 18:59:54.461642 14015 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:41623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.2.keytab
--principal=kudu/127.13.109.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.109.2
--webserver_port=0
--tserver_master_addrs=127.13.109.62:34281
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14015
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.2
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:59:54.462811 14015 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:59:54.463840 14015 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:59:54.473519 14020 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:54.473459 14021 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:54.473467 14023 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:59:54.473874 14015 server_base.cc:1034] running on GCE node
I20260811 18:59:54.474337 14015 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:59:54.474973 14015 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:59:54.476193 14015 hybrid_clock.cc:648] HybridClock initialized: now 1786474794476179 us; error 29 us; skew 500 ppm
Aug 11 18:59:54 dist-test-slave-0kls krb5kdc[13755](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474794, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:59:54.483100 14015 init.cc:377] Logged in from keytab as kudu/127.13.109.2@KRBTEST.COM (short username kudu)
I20260811 18:59:54.484018 14015 webserver.cc:460] Webserver started at http://127.13.109.2:33131/ using document root <none> and password file <none>
I20260811 18:59:54.484596 14015 fs_manager.cc:359] Metadata directory not provided
I20260811 18:59:54.484660 14015 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:59:54.484841 14015 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:59:54.487057 14015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/minicluster-data/ts-1/data/instance:
uuid: "4cee9df1de5e46fa9af661e752cd4d9c"
format_stamp: "Formatted at 2026-08-11 18:59:54 on dist-test-slave-0kls"
server_key: "0c47d7944fcd08f95537e72b86dbe743"
server_key_iv: "d21845b4f482df6579600030ad98a441"
server_key_version: "encryptionkey@0"
I20260811 18:59:54.487593 14015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/minicluster-data/ts-1/wal/instance:
uuid: "4cee9df1de5e46fa9af661e752cd4d9c"
format_stamp: "Formatted at 2026-08-11 18:59:54 on dist-test-slave-0kls"
server_key: "0c47d7944fcd08f95537e72b86dbe743"
server_key_iv: "d21845b4f482df6579600030ad98a441"
server_key_version: "encryptionkey@0"
I20260811 18:59:54.491652 14015 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260811 18:59:54.494334 14031 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:54.495728 14015 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:59:54.495887 14015 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/minicluster-data/ts-1/wal
uuid: "4cee9df1de5e46fa9af661e752cd4d9c"
format_stamp: "Formatted at 2026-08-11 18:59:54 on dist-test-slave-0kls"
server_key: "0c47d7944fcd08f95537e72b86dbe743"
server_key_iv: "d21845b4f482df6579600030ad98a441"
server_key_version: "encryptionkey@0"
I20260811 18:59:54.496052 14015 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:59:54.511992 14015 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:59:54.515219 14015 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:59:54.515488 14015 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:59:54.516230 14015 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:59:54.517225 14015 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:59:54.517280 14015 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:54.517359 14015 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:59:54.517400 14015 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:54.528278 14015 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.2:36159
I20260811 18:59:54.528298 14157 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.2:36159 every 8 connection(s)
I20260811 18:59:54.529278 14015 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
I20260811 18:59:54.530197 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 14015
I20260811 18:59:54.530297 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/minicluster-data/ts-1/wal/instance
I20260811 18:59:54.530505 13748 external_mini_cluster.cc:1428] Setting key 266dfdbe65e722d37f1dcd01acf1cd69
WARNING: no policy specified for kudu/127.13.109.3@KRBTEST.COM; defaulting to no policy
Aug 11 18:59:54 dist-test-slave-0kls krb5kdc[13755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474794, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.2@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.109.3@KRBTEST.COM" created.
I20260811 18:59:54.545539 13844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:54.532127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.2:50467 (local address 127.13.109.62:34281)
0811 18:59:54.532266 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:54.532271 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:54.533368 (+ 1097us) server_negotiation.cc:407] Connection header received
0811 18:59:54.534867 (+ 1499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:54.534871 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:54.534931 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:54.535056 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:54.536332 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:54.537007 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:54.538267 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:54.538511 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:54.538635 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:54.541371 (+ 2736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:54.541374 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:54.541376 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:59:54.541401 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:54.543131 (+ 1730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:54.543748 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:54.543752 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:54.543754 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:54.543842 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:54.544249 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:54.544251 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:54.544253 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:54.544428 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:54.544504 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:54.545146 (+ 642us) server_negotiation.cc:299] Negotiation successful
0811 18:59:54.545310 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 18:59:54.546476 14161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:54.532608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:34281 (local address 127.13.109.2:50467)
0811 18:59:54.533192 (+ 584us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:54.533247 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 18:59:54.534642 (+ 1395us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:59:54.535282 (+ 640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:59:54.535295 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:59:54.535686 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:59:54.536170 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:54.536185 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:54.537152 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:54.537162 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:54.538131 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:54.538142 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:54.538678 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:54.538683 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:54.538865 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:54.539516 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:59:54.539532 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:59:54.541204 (+ 1672us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:59:54.543296 (+ 2092us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:59:54.543304 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:59:54.543324 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:59:54.543647 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:59:54.543940 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:59:54.543946 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:59:54.543951 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:59:54.544148 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:59:54.544531 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:59:54.544540 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:59:54.544890 (+ 350us) client_negotiation.cc:769] Sending connection context
0811 18:59:54.545112 (+ 222us) client_negotiation.cc:240] Negotiation successful
0811 18:59:54.545483 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":206,"threads_started":1}
I20260811 18:59:54.547807 14158 heartbeater.cc:347] Connected to a master server at 127.13.109.62:34281
I20260811 18:59:54.548079 14158 heartbeater.cc:464] Registering TS with master...
I20260811 18:59:54.548621 14158 heartbeater.cc:511] Master 127.13.109.62:34281 requested a full tablet report, sending...
I20260811 18:59:54.549831 13805 ts_manager.cc:194] Registered new tserver with Master: 4cee9df1de5e46fa9af661e752cd4d9c (127.13.109.2:36159)
I20260811 18:59:54.550652 13805 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.109.2@KRBTEST.COM'} at 127.13.109.2:50467
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.109.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.3.keytab.
Entry for principal kudu/127.13.109.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.3.keytab.
I20260811 18:59:54.561992 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.3:0
--local_ip_for_outbound_sockets=127.13.109.3
--webserver_interface=127.13.109.3
--webserver_port=0
--tserver_master_addrs=127.13.109.62:34281
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.3.keytab
--principal=kudu/127.13.109.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:41623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:59:54.698112 14164 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:59:54.698376 14164 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:59:54.698421 14164 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:59:54.702757 14164 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:59:54.702831 14164 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:59:54.702893 14164 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.3
I20260811 18:59:54.708273 14164 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:41623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.3.keytab
--principal=kudu/127.13.109.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.109.3
--webserver_port=0
--tserver_master_addrs=127.13.109.62:34281
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14164
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.3
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:59:54.709509 14164 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:59:54.710623 14164 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:59:54.721197 14173 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:54.721311 14171 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:54.721212 14170 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:59:54.721455 14164 server_base.cc:1034] running on GCE node
I20260811 18:59:54.722002 14164 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:59:54.722699 14164 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:59:54.723901 14164 hybrid_clock.cc:648] HybridClock initialized: now 1786474794723879 us; error 43 us; skew 500 ppm
Aug 11 18:59:54 dist-test-slave-0kls krb5kdc[13755](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474794, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:59:54.727038 14164 init.cc:377] Logged in from keytab as kudu/127.13.109.3@KRBTEST.COM (short username kudu)
I20260811 18:59:54.728039 14164 webserver.cc:460] Webserver started at http://127.13.109.3:44719/ using document root <none> and password file <none>
I20260811 18:59:54.728679 14164 fs_manager.cc:359] Metadata directory not provided
I20260811 18:59:54.728780 14164 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:59:54.729040 14164 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:59:54.731420 14164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/minicluster-data/ts-2/data/instance:
uuid: "27b6da8c2fef4d4aa94129d461636ebb"
format_stamp: "Formatted at 2026-08-11 18:59:54 on dist-test-slave-0kls"
server_key: "d639d1ad362f64f8f34a9dfbad300e17"
server_key_iv: "447b50e9f47051ac861a2e68d78ff966"
server_key_version: "encryptionkey@0"
I20260811 18:59:54.732043 14164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/minicluster-data/ts-2/wal/instance:
uuid: "27b6da8c2fef4d4aa94129d461636ebb"
format_stamp: "Formatted at 2026-08-11 18:59:54 on dist-test-slave-0kls"
server_key: "d639d1ad362f64f8f34a9dfbad300e17"
server_key_iv: "447b50e9f47051ac861a2e68d78ff966"
server_key_version: "encryptionkey@0"
I20260811 18:59:54.736126 14164 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 18:59:54.738760 14180 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:54.740152 14164 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 18:59:54.740302 14164 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/minicluster-data/ts-2/wal
uuid: "27b6da8c2fef4d4aa94129d461636ebb"
format_stamp: "Formatted at 2026-08-11 18:59:54 on dist-test-slave-0kls"
server_key: "d639d1ad362f64f8f34a9dfbad300e17"
server_key_iv: "447b50e9f47051ac861a2e68d78ff966"
server_key_version: "encryptionkey@0"
I20260811 18:59:54.740428 14164 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:59:54.785920 14164 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:59:54.788971 14164 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:59:54.789199 14164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:59:54.789858 14164 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:59:54.790808 14164 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:59:54.790882 14164 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:54.790957 14164 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:59:54.791002 14164 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:54.801178 14164 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.3:45135
I20260811 18:59:54.801245 14302 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.3:45135 every 8 connection(s)
I20260811 18:59:54.802194 14164 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
I20260811 18:59:54.810770 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 14164
I20260811 18:59:54.810874 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474792496492-13748-0/minicluster-data/ts-2/wal/instance
I20260811 18:59:54.811199 13748 external_mini_cluster.cc:1428] Setting key fc13fb871c054ed2d960b7d1871a243d
Aug 11 18:59:54 dist-test-slave-0kls krb5kdc[13755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474794, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.3@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
I20260811 18:59:54.816893 13844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:54.804147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.3:58877 (local address 127.13.109.62:34281)
0811 18:59:54.804304 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:54.804308 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:54.805118 (+ 810us) server_negotiation.cc:407] Connection header received
0811 18:59:54.806545 (+ 1427us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:54.806548 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:54.806614 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:54.806726 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:54.807920 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:54.808457 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:54.809674 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:54.809947 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:54.810057 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:54.812753 (+ 2696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:54.812757 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:54.812760 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:59:54.812787 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:54.814492 (+ 1705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:54.815145 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:54.815152 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:54.815155 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:54.815266 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:54.815657 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:54.815665 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:54.815666 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:54.815931 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:54.816034 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:54.816582 (+ 548us) server_negotiation.cc:299] Negotiation successful
0811 18:59:54.816727 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 18:59:54.817835 14306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:54.804494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:34281 (local address 127.13.109.3:58877)
0811 18:59:54.804953 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:54.804998 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 18:59:54.806327 (+ 1329us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:59:54.806903 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:59:54.806910 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:59:54.807279 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:59:54.807765 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:54.807778 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:54.808594 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:54.808602 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:54.809535 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:54.809545 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:54.810117 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:54.810122 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:54.810294 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:54.811016 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:59:54.811053 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:59:54.812596 (+ 1543us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:59:54.814624 (+ 2028us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:59:54.814631 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:59:54.814648 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:59:54.815040 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:59:54.815365 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:59:54.815368 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:59:54.815370 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:59:54.815523 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:59:54.816050 (+ 527us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:59:54.816058 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:59:54.816339 (+ 281us) client_negotiation.cc:769] Sending connection context
0811 18:59:54.816542 (+ 203us) client_negotiation.cc:240] Negotiation successful
0811 18:59:54.816901 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":127,"threads_started":1}
I20260811 18:59:54.819100 14304 heartbeater.cc:347] Connected to a master server at 127.13.109.62:34281
I20260811 18:59:54.819372 14304 heartbeater.cc:464] Registering TS with master...
I20260811 18:59:54.819906 14304 heartbeater.cc:511] Master 127.13.109.62:34281 requested a full tablet report, sending...
I20260811 18:59:54.821033 13805 ts_manager.cc:194] Registered new tserver with Master: 27b6da8c2fef4d4aa94129d461636ebb (127.13.109.3:45135)
I20260811 18:59:54.821585 13805 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.109.3@KRBTEST.COM'} at 127.13.109.3:58877
I20260811 18:59:54.826133 13748 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 18:59:54 dist-test-slave-0kls krb5kdc[13755](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474794, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 18:59:54 dist-test-slave-0kls krb5kdc[13755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474794, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.13.109.1@KRBTEST.COM
I20260811 18:59:54.855023 14311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:54.845194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41254 (local address 127.13.109.1:43965)
0811 18:59:54.845568 (+ 374us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:54.845574 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:54.845602 (+ 28us) server_negotiation.cc:407] Connection header received
0811 18:59:54.845750 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:54.845761 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:54.846029 (+ 268us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:54.846179 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:54.846866 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:54.847641 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:54.848438 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:54.848820 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:54.848916 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:54.851494 (+ 2578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:54.851505 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:54.851520 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:59:54.851545 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:54.853145 (+ 1600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:54.853555 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:54.853560 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:54.853561 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:54.853642 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:54.854064 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:54.854066 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:54.854067 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:54.854352 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:54.854512 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:54.854739 (+ 227us) server_negotiation.cc:299] Negotiation successful
0811 18:59:54.854872 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":249,"thread_start_us":101,"threads_started":1}
W20260811 18:59:54.856283 13913 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41254
Aug 11 18:59:54 dist-test-slave-0kls krb5kdc[13755](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474794, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 18:59:54 dist-test-slave-0kls krb5kdc[13755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474794, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.109.1@KRBTEST.COM
I20260811 18:59:54.885860 14311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:54.877100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41266 (local address 127.13.109.1:43965)
0811 18:59:54.877284 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:54.877289 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:54.877424 (+ 135us) server_negotiation.cc:407] Connection header received
0811 18:59:54.877618 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:54.877622 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:54.877673 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:54.877762 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:54.878566 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:54.879080 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:54.879961 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:54.880210 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:54.880323 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:54.882847 (+ 2524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:54.882853 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:54.882860 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:59:54.882885 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:54.884095 (+ 1210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:54.884547 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:54.884552 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:54.884554 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:54.884641 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:54.884930 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:54.884933 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:54.884935 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:54.885109 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:54.885279 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:54.885525 (+ 246us) server_negotiation.cc:299] Negotiation successful
0811 18:59:54.885652 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 18:59:54.886947 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 13869
I20260811 18:59:54.896196 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 14015
I20260811 18:59:54.904747 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 14164
I20260811 18:59:54.911855 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 13770
2026-08-11T18:59:54Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2409 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:59:54 dist-test-slave-0kls krb5kdc[14318](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:59:54 dist-test-slave-0kls krb5kdc[14318](info): set up 2 sockets
Aug 11 18:59:54 dist-test-slave-0kls krb5kdc[14318](info): commencing operation
krb5kdc: starting...
W20260811 18:59:56.014273 13748 mini_kdc.cc:121] Time spent starting KDC: real 1.076s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/krb5kdc/test-user.keytab.
Aug 11 18:59:56 dist-test-slave-0kls krb5kdc[14318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474796, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:59:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:59:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.109.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.109.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.109.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab.
Entry for principal kudu/127.13.109.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab.
I20260811 18:59:56.153618 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.62:43179
--webserver_interface=127.13.109.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab
--principal=kudu/127.13.109.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:36785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.109.62:43179
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:59:56.273653 14333 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:59:56.274026 14333 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:59:56.274109 14333 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:59:56.278311 14333 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:59:56.278437 14333 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:59:56.278479 14333 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:59:56.278523 14333 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:59:56.278559 14333 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:59:56.284252 14333 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:36785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.109.62:43179
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab
--principal=kudu/127.13.109.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.62:43179
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.109.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14333
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:59:56.285476 14333 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:59:56.286621 14333 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:59:56.295883 14342 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:56.295892 14340 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:56.295892 14339 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:59:56.295933 14333 server_base.cc:1034] running on GCE node
I20260811 18:59:56.296675 14333 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:59:56.297900 14333 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:59:56.299094 14333 hybrid_clock.cc:648] HybridClock initialized: now 1786474796299073 us; error 38 us; skew 500 ppm
Aug 11 18:59:56 dist-test-slave-0kls krb5kdc[14318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474796, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:59:56.301919 14333 init.cc:377] Logged in from keytab as kudu/127.13.109.62@KRBTEST.COM (short username kudu)
I20260811 18:59:56.302906 14333 webserver.cc:460] Webserver started at http://127.13.109.62:35089/ using document root <none> and password file <none>
I20260811 18:59:56.303507 14333 fs_manager.cc:359] Metadata directory not provided
I20260811 18:59:56.303607 14333 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:59:56.303876 14333 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:59:56.306128 14333 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/minicluster-data/master-0/data/instance:
uuid: "38fd6f52fd13414188b0415b4230daff"
format_stamp: "Formatted at 2026-08-11 18:59:56 on dist-test-slave-0kls"
server_key: "66628b2c1bf51e99f91e16e7f3c5ed2d"
server_key_iv: "1ffdc42084f7a88567cff0d8bbe9d300"
server_key_version: "encryptionkey@0"
I20260811 18:59:56.306743 14333 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/minicluster-data/master-0/wal/instance:
uuid: "38fd6f52fd13414188b0415b4230daff"
format_stamp: "Formatted at 2026-08-11 18:59:56 on dist-test-slave-0kls"
server_key: "66628b2c1bf51e99f91e16e7f3c5ed2d"
server_key_iv: "1ffdc42084f7a88567cff0d8bbe9d300"
server_key_version: "encryptionkey@0"
I20260811 18:59:56.310873 14333 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 18:59:56.313462 14349 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:56.314867 14333 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 18:59:56.315022 14333 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/minicluster-data/master-0/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/minicluster-data/master-0/wal
uuid: "38fd6f52fd13414188b0415b4230daff"
format_stamp: "Formatted at 2026-08-11 18:59:56 on dist-test-slave-0kls"
server_key: "66628b2c1bf51e99f91e16e7f3c5ed2d"
server_key_iv: "1ffdc42084f7a88567cff0d8bbe9d300"
server_key_version: "encryptionkey@0"
I20260811 18:59:56.315153 14333 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:59:56.344369 14333 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:59:56.348246 14333 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:59:56.348488 14333 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:59:56.356436 14333 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.62:43179
I20260811 18:59:56.356457 14405 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.62:43179 every 8 connection(s)
I20260811 18:59:56.357595 14333 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
I20260811 18:59:56.360947 14406 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:59:56.362104 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 14333
I20260811 18:59:56.362202 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/minicluster-data/master-0/wal/instance
I20260811 18:59:56.362502 13748 external_mini_cluster.cc:1428] Setting key 4c48a10631df34b3d3343ccdd9efc707
I20260811 18:59:56.367734 14406 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38fd6f52fd13414188b0415b4230daff: Bootstrap starting.
I20260811 18:59:56.370467 14406 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 38fd6f52fd13414188b0415b4230daff: Neither blocks nor log segments found. Creating new log.
I20260811 18:59:56.371325 14406 log.cc:824] T 00000000000000000000000000000000 P 38fd6f52fd13414188b0415b4230daff: Log is configured to *not* fsync() on all Append() calls
Aug 11 18:59:56 dist-test-slave-0kls krb5kdc[14318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474796, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
I20260811 18:59:56.374030 14406 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38fd6f52fd13414188b0415b4230daff: No bootstrap required, opened a new log
I20260811 18:59:56.378206 14409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:56.364112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52030 (local address 127.13.109.62:43179)
0811 18:59:56.364607 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:56.364623 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:56.364662 (+ 39us) server_negotiation.cc:407] Connection header received
0811 18:59:56.365355 (+ 693us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:56.365374 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:56.365733 (+ 359us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:56.366167 (+ 434us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:56.367018 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:56.368106 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:56.369005 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:56.369612 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:56.369835 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:56.372818 (+ 2983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:56.372824 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:56.372840 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:59:56.372872 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:56.374945 (+ 2073us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:56.375495 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:56.375501 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:56.375505 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:56.375632 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:56.375966 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:56.375968 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:56.375970 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:56.376424 (+ 454us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:56.376594 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:56.376913 (+ 319us) server_negotiation.cc:299] Negotiation successful
0811 18:59:56.377296 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":140,"threads_started":1}
I20260811 18:59:56.378435 14406 raft_consensus.cc:348] T 00000000000000000000000000000000 P 38fd6f52fd13414188b0415b4230daff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38fd6f52fd13414188b0415b4230daff" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 43179 } }
I20260811 18:59:56.378703 14406 raft_consensus.cc:374] T 00000000000000000000000000000000 P 38fd6f52fd13414188b0415b4230daff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:59:56.378782 14406 raft_consensus.cc:729] T 00000000000000000000000000000000 P 38fd6f52fd13414188b0415b4230daff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38fd6f52fd13414188b0415b4230daff, State: Initialized, Role: FOLLOWER
I20260811 18:59:56.379281 14406 consensus_queue.cc:260] T 00000000000000000000000000000000 P 38fd6f52fd13414188b0415b4230daff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38fd6f52fd13414188b0415b4230daff" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 43179 } }
I20260811 18:59:56.379441 14406 raft_consensus.cc:388] T 00000000000000000000000000000000 P 38fd6f52fd13414188b0415b4230daff [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:59:56.379529 14406 raft_consensus.cc:482] T 00000000000000000000000000000000 P 38fd6f52fd13414188b0415b4230daff [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:59:56.379637 14406 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 38fd6f52fd13414188b0415b4230daff [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:59:56.380652 14406 raft_consensus.cc:504] T 00000000000000000000000000000000 P 38fd6f52fd13414188b0415b4230daff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38fd6f52fd13414188b0415b4230daff" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 43179 } }
I20260811 18:59:56.381022 14406 leader_election.cc:302] T 00000000000000000000000000000000 P 38fd6f52fd13414188b0415b4230daff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 38fd6f52fd13414188b0415b4230daff; no voters:
I20260811 18:59:56.381359 14406 leader_election.cc:288] T 00000000000000000000000000000000 P 38fd6f52fd13414188b0415b4230daff [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:59:56.381531 14411 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 38fd6f52fd13414188b0415b4230daff [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:59:56.381775 14411 raft_consensus.cc:686] T 00000000000000000000000000000000 P 38fd6f52fd13414188b0415b4230daff [term 1 LEADER]: Becoming Leader. State: Replica: 38fd6f52fd13414188b0415b4230daff, State: Running, Role: LEADER
I20260811 18:59:56.382145 14411 consensus_queue.cc:237] T 00000000000000000000000000000000 P 38fd6f52fd13414188b0415b4230daff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38fd6f52fd13414188b0415b4230daff" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 43179 } }
I20260811 18:59:56.382856 14406 sys_catalog.cc:564] T 00000000000000000000000000000000 P 38fd6f52fd13414188b0415b4230daff [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:59:56.384075 14412 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38fd6f52fd13414188b0415b4230daff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "38fd6f52fd13414188b0415b4230daff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38fd6f52fd13414188b0415b4230daff" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 43179 } } }
I20260811 18:59:56.384232 14412 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38fd6f52fd13414188b0415b4230daff [sys.catalog]: This master's current role is: LEADER
I20260811 18:59:56.384531 14413 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38fd6f52fd13414188b0415b4230daff [sys.catalog]: SysCatalogTable state changed. Reason: New leader 38fd6f52fd13414188b0415b4230daff. Latest consensus state: current_term: 1 leader_uuid: "38fd6f52fd13414188b0415b4230daff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38fd6f52fd13414188b0415b4230daff" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 43179 } } }
I20260811 18:59:56.384616 14421 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:59:56.384646 14413 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38fd6f52fd13414188b0415b4230daff [sys.catalog]: This master's current role is: LEADER
I20260811 18:59:56.388084 14421 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:59:56.394737 14421 catalog_manager.cc:1347] Generated new cluster ID: 0600a97f572d460e9cc60b8ee272d632
I20260811 18:59:56.394824 14421 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:59:56.405824 14421 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:59:56.407202 14421 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:59:56.415709 14421 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 38fd6f52fd13414188b0415b4230daff: Generated new TSK 0
I20260811 18:59:56.416707 14421 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.109.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.109.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.109.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.1.keytab.
Entry for principal kudu/127.13.109.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.1.keytab.
I20260811 18:59:56.449424 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.1:0
--local_ip_for_outbound_sockets=127.13.109.1
--webserver_interface=127.13.109.1
--webserver_port=0
--tserver_master_addrs=127.13.109.62:43179
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.1.keytab
--principal=kudu/127.13.109.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:36785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:59:56.570570 14434 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:59:56.570895 14434 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:59:56.570942 14434 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:59:56.575243 14434 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:59:56.575317 14434 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:59:56.575383 14434 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.1
I20260811 18:59:56.581120 14434 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:36785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.1.keytab
--principal=kudu/127.13.109.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.109.1
--webserver_port=0
--tserver_master_addrs=127.13.109.62:43179
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14434
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.1
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:59:56.582341 14434 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:59:56.583482 14434 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:59:56.593874 14443 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:56.593878 14440 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:56.593878 14439 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:59:56.594234 14434 server_base.cc:1034] running on GCE node
I20260811 18:59:56.594674 14434 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:59:56.595335 14434 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:59:56.596513 14434 hybrid_clock.cc:648] HybridClock initialized: now 1786474796596496 us; error 29 us; skew 500 ppm
Aug 11 18:59:56 dist-test-slave-0kls krb5kdc[14318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474796, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:59:56.599561 14434 init.cc:377] Logged in from keytab as kudu/127.13.109.1@KRBTEST.COM (short username kudu)
I20260811 18:59:56.600590 14434 webserver.cc:460] Webserver started at http://127.13.109.1:34379/ using document root <none> and password file <none>
I20260811 18:59:56.601198 14434 fs_manager.cc:359] Metadata directory not provided
I20260811 18:59:56.601295 14434 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:59:56.601545 14434 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:59:56.603715 14434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/minicluster-data/ts-0/data/instance:
uuid: "24f54c31fefe4d938e63c41a6026ce55"
format_stamp: "Formatted at 2026-08-11 18:59:56 on dist-test-slave-0kls"
server_key: "91fa133f4d1eb969a7fbc05e8d770683"
server_key_iv: "dd6a756757b5fda46c93606a12057775"
server_key_version: "encryptionkey@0"
I20260811 18:59:56.604313 14434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/minicluster-data/ts-0/wal/instance:
uuid: "24f54c31fefe4d938e63c41a6026ce55"
format_stamp: "Formatted at 2026-08-11 18:59:56 on dist-test-slave-0kls"
server_key: "91fa133f4d1eb969a7fbc05e8d770683"
server_key_iv: "dd6a756757b5fda46c93606a12057775"
server_key_version: "encryptionkey@0"
I20260811 18:59:56.608235 14434 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260811 18:59:56.610836 14451 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:56.612075 14434 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 18:59:56.612219 14434 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/minicluster-data/ts-0/wal
uuid: "24f54c31fefe4d938e63c41a6026ce55"
format_stamp: "Formatted at 2026-08-11 18:59:56 on dist-test-slave-0kls"
server_key: "91fa133f4d1eb969a7fbc05e8d770683"
server_key_iv: "dd6a756757b5fda46c93606a12057775"
server_key_version: "encryptionkey@0"
I20260811 18:59:56.612340 14434 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:59:56.628540 14434 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:59:56.631775 14434 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:59:56.631994 14434 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:59:56.632637 14434 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:59:56.633534 14434 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:59:56.633616 14434 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:56.633708 14434 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:59:56.633762 14434 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:56.643590 14434 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.1:46169
I20260811 18:59:56.643622 14575 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.1:46169 every 8 connection(s)
I20260811 18:59:56.644629 14434 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
I20260811 18:59:56.647485 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 14434
I20260811 18:59:56.647620 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/minicluster-data/ts-0/wal/instance
I20260811 18:59:56.647943 13748 external_mini_cluster.cc:1428] Setting key bbd03915673493438dd1ea74a75d2ca9
Aug 11 18:59:56 dist-test-slave-0kls krb5kdc[14318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474796, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.1@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.109.2@KRBTEST.COM; defaulting to no policy
I20260811 18:59:56.660068 14409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:56.646790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.1:55709 (local address 127.13.109.62:43179)
0811 18:59:56.646945 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:56.646949 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:56.648248 (+ 1299us) server_negotiation.cc:407] Connection header received
0811 18:59:56.649206 (+ 958us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:56.649210 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:56.649269 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:56.649379 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:56.650735 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:56.651270 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:56.652571 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:56.652852 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:56.652939 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:56.655949 (+ 3010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:56.655955 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:56.655961 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:59:56.656003 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:56.657756 (+ 1753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:56.658329 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:56.658333 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:56.658335 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:56.658430 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:56.658786 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:56.658792 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:56.658796 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:56.659077 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:56.659192 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:56.659811 (+ 619us) server_negotiation.cc:299] Negotiation successful
0811 18:59:56.659934 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 18:59:56.660986 14578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:56.647195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:43179 (local address 127.13.109.1:55709)
0811 18:59:56.648039 (+ 844us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:56.648104 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0811 18:59:56.648974 (+ 870us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:59:56.649572 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:59:56.649580 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:59:56.650044 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:59:56.650566 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:56.650581 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:56.651472 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:56.651479 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:56.652431 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:56.652443 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:56.653007 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:56.653009 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:56.653161 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:56.653876 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:59:56.653894 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:59:56.655719 (+ 1825us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:59:56.657922 (+ 2203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:59:56.657927 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:59:56.657942 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:59:56.658215 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:59:56.658535 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:59:56.658537 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:59:56.658538 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:59:56.658680 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:59:56.659193 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:59:56.659198 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:59:56.659514 (+ 316us) client_negotiation.cc:769] Sending connection context
0811 18:59:56.659706 (+ 192us) client_negotiation.cc:240] Negotiation successful
0811 18:59:56.660019 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":599,"thread_start_us":125,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.109.2@KRBTEST.COM" created.
I20260811 18:59:56.662593 14576 heartbeater.cc:347] Connected to a master server at 127.13.109.62:43179
I20260811 18:59:56.662873 14576 heartbeater.cc:464] Registering TS with master...
I20260811 18:59:56.663435 14576 heartbeater.cc:511] Master 127.13.109.62:43179 requested a full tablet report, sending...
I20260811 18:59:56.665081 14367 ts_manager.cc:194] Registered new tserver with Master: 24f54c31fefe4d938e63c41a6026ce55 (127.13.109.1:46169)
I20260811 18:59:56.666781 14367 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.109.1@KRBTEST.COM'} at 127.13.109.1:55709
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.109.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.2.keytab.
Entry for principal kudu/127.13.109.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.2.keytab.
I20260811 18:59:56.682075 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.2:0
--local_ip_for_outbound_sockets=127.13.109.2
--webserver_interface=127.13.109.2
--webserver_port=0
--tserver_master_addrs=127.13.109.62:43179
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.2.keytab
--principal=kudu/127.13.109.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:36785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:59:56.802547 14581 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:59:56.802899 14581 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:59:56.802999 14581 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:59:56.807225 14581 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:59:56.807319 14581 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:59:56.807406 14581 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.2
I20260811 18:59:56.813023 14581 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:36785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.2.keytab
--principal=kudu/127.13.109.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.109.2
--webserver_port=0
--tserver_master_addrs=127.13.109.62:43179
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14581
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.2
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:59:56.814248 14581 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:59:56.815392 14581 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:59:56.825835 14588 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:56.825763 14587 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:59:56.825763 14581 server_base.cc:1034] running on GCE node
W20260811 18:59:56.825767 14590 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:59:56.826615 14581 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:59:56.827314 14581 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:59:56.828531 14581 hybrid_clock.cc:648] HybridClock initialized: now 1786474796828508 us; error 37 us; skew 500 ppm
Aug 11 18:59:56 dist-test-slave-0kls krb5kdc[14318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474796, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:59:56.831400 14581 init.cc:377] Logged in from keytab as kudu/127.13.109.2@KRBTEST.COM (short username kudu)
I20260811 18:59:56.832294 14581 webserver.cc:460] Webserver started at http://127.13.109.2:37757/ using document root <none> and password file <none>
I20260811 18:59:56.832894 14581 fs_manager.cc:359] Metadata directory not provided
I20260811 18:59:56.832989 14581 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:59:56.833257 14581 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:59:56.835510 14581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/minicluster-data/ts-1/data/instance:
uuid: "42cb7e3a86ca46e8ae5c70d28f1db3b6"
format_stamp: "Formatted at 2026-08-11 18:59:56 on dist-test-slave-0kls"
server_key: "6ca1532aa8cd80a28f0fff1ae3609c51"
server_key_iv: "2219f0078fdd116a770a9efae098fe09"
server_key_version: "encryptionkey@0"
I20260811 18:59:56.836149 14581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/minicluster-data/ts-1/wal/instance:
uuid: "42cb7e3a86ca46e8ae5c70d28f1db3b6"
format_stamp: "Formatted at 2026-08-11 18:59:56 on dist-test-slave-0kls"
server_key: "6ca1532aa8cd80a28f0fff1ae3609c51"
server_key_iv: "2219f0078fdd116a770a9efae098fe09"
server_key_version: "encryptionkey@0"
I20260811 18:59:56.840847 14581 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 18:59:56.843623 14597 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:56.844847 14581 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 18:59:56.844992 14581 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/minicluster-data/ts-1/wal
uuid: "42cb7e3a86ca46e8ae5c70d28f1db3b6"
format_stamp: "Formatted at 2026-08-11 18:59:56 on dist-test-slave-0kls"
server_key: "6ca1532aa8cd80a28f0fff1ae3609c51"
server_key_iv: "2219f0078fdd116a770a9efae098fe09"
server_key_version: "encryptionkey@0"
I20260811 18:59:56.845119 14581 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:59:56.860302 14581 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:59:56.863555 14581 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:59:56.863785 14581 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:59:56.864498 14581 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:59:56.865399 14581 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:59:56.865449 14581 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:56.865521 14581 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:59:56.865554 14581 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:56.875176 14581 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.2:43341
I20260811 18:59:56.875196 14715 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.2:43341 every 8 connection(s)
I20260811 18:59:56.876204 14581 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
I20260811 18:59:56.880156 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 14581
I20260811 18:59:56.880299 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/minicluster-data/ts-1/wal/instance
I20260811 18:59:56.880606 13748 external_mini_cluster.cc:1428] Setting key 468b790082e7aa88a525d530c94ab67b
Aug 11 18:59:56 dist-test-slave-0kls krb5kdc[14318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474796, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.2@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.109.3@KRBTEST.COM; defaulting to no policy
I20260811 18:59:56.892402 14409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:56.878524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.2:48517 (local address 127.13.109.62:43179)
0811 18:59:56.878685 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:56.878689 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:56.879666 (+ 977us) server_negotiation.cc:407] Connection header received
0811 18:59:56.880828 (+ 1162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:56.880831 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:56.880904 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:56.880998 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:56.882310 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:56.882839 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:56.884186 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:56.884544 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:56.884649 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:56.887887 (+ 3238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:56.887891 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:56.887894 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:59:56.887930 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:56.890029 (+ 2099us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:56.890599 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:56.890603 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:56.890607 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:56.890712 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:56.891065 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:56.891070 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:56.891072 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:56.891347 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:56.891458 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:56.892034 (+ 576us) server_negotiation.cc:299] Negotiation successful
0811 18:59:56.892162 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 18:59:56.893415 14718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:56.878992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:43179 (local address 127.13.109.2:48517)
0811 18:59:56.879498 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:56.879545 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 18:59:56.880564 (+ 1019us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:59:56.881195 (+ 631us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:59:56.881202 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:59:56.881572 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:59:56.882162 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:56.882178 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:56.883010 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:56.883013 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:56.883930 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:56.883940 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:56.884730 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:56.884736 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:56.884988 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:56.885764 (+ 776us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:59:56.885810 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:59:56.887666 (+ 1856us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:59:56.890169 (+ 2503us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:59:56.890176 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:59:56.890192 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:59:56.890479 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:59:56.890817 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:59:56.890819 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:59:56.890820 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:59:56.890965 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:59:56.891462 (+ 497us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:59:56.891468 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:59:56.891775 (+ 307us) client_negotiation.cc:769] Sending connection context
0811 18:59:56.891945 (+ 170us) client_negotiation.cc:240] Negotiation successful
0811 18:59:56.892275 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":137,"threads_started":1}
I20260811 18:59:56.894694 14716 heartbeater.cc:347] Connected to a master server at 127.13.109.62:43179
I20260811 18:59:56.894989 14716 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.109.3@KRBTEST.COM" created.
I20260811 18:59:56.895627 14716 heartbeater.cc:511] Master 127.13.109.62:43179 requested a full tablet report, sending...
I20260811 18:59:56.896858 14367 ts_manager.cc:194] Registered new tserver with Master: 42cb7e3a86ca46e8ae5c70d28f1db3b6 (127.13.109.2:43341)
I20260811 18:59:56.897830 14367 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.109.2@KRBTEST.COM'} at 127.13.109.2:48517
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.109.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.3.keytab.
Entry for principal kudu/127.13.109.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.3.keytab.
I20260811 18:59:56.913863 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.3:0
--local_ip_for_outbound_sockets=127.13.109.3
--webserver_interface=127.13.109.3
--webserver_port=0
--tserver_master_addrs=127.13.109.62:43179
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.3.keytab
--principal=kudu/127.13.109.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:36785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:59:57.034991 14722 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:59:57.035317 14722 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:59:57.035398 14722 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:59:57.039711 14722 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:59:57.039820 14722 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:59:57.039888 14722 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.3
I20260811 18:59:57.045843 14722 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:36785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.3.keytab
--principal=kudu/127.13.109.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.109.3
--webserver_port=0
--tserver_master_addrs=127.13.109.62:43179
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14722
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.3
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:59:57.047130 14722 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:59:57.048375 14722 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:59:57.059278 14729 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:57.059278 14732 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:57.059296 14728 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:59:57.059552 14722 server_base.cc:1034] running on GCE node
I20260811 18:59:57.060256 14722 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:59:57.060922 14722 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:59:57.062083 14722 hybrid_clock.cc:648] HybridClock initialized: now 1786474797062070 us; error 26 us; skew 500 ppm
Aug 11 18:59:57 dist-test-slave-0kls krb5kdc[14318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474797, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:59:57.065039 14722 init.cc:377] Logged in from keytab as kudu/127.13.109.3@KRBTEST.COM (short username kudu)
I20260811 18:59:57.066160 14722 webserver.cc:460] Webserver started at http://127.13.109.3:33723/ using document root <none> and password file <none>
I20260811 18:59:57.066792 14722 fs_manager.cc:359] Metadata directory not provided
I20260811 18:59:57.066864 14722 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:59:57.067170 14722 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:59:57.069514 14722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/minicluster-data/ts-2/data/instance:
uuid: "23adebc4e4204ca1978d788c9c5222b5"
format_stamp: "Formatted at 2026-08-11 18:59:57 on dist-test-slave-0kls"
server_key: "dd885f90fa66f27cddcfafee6de54d4d"
server_key_iv: "61a340ee759a2dcf6a39b9fd4eec22bc"
server_key_version: "encryptionkey@0"
I20260811 18:59:57.070226 14722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/minicluster-data/ts-2/wal/instance:
uuid: "23adebc4e4204ca1978d788c9c5222b5"
format_stamp: "Formatted at 2026-08-11 18:59:57 on dist-test-slave-0kls"
server_key: "dd885f90fa66f27cddcfafee6de54d4d"
server_key_iv: "61a340ee759a2dcf6a39b9fd4eec22bc"
server_key_version: "encryptionkey@0"
I20260811 18:59:57.074401 14722 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 18:59:57.077092 14740 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:57.078554 14722 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:59:57.078673 14722 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/minicluster-data/ts-2/wal
uuid: "23adebc4e4204ca1978d788c9c5222b5"
format_stamp: "Formatted at 2026-08-11 18:59:57 on dist-test-slave-0kls"
server_key: "dd885f90fa66f27cddcfafee6de54d4d"
server_key_iv: "61a340ee759a2dcf6a39b9fd4eec22bc"
server_key_version: "encryptionkey@0"
I20260811 18:59:57.078778 14722 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:59:57.103813 14722 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:59:57.107460 14722 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:59:57.107637 14722 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:59:57.108268 14722 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:59:57.109197 14722 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:59:57.109247 14722 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:57.109288 14722 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:59:57.109304 14722 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:57.120311 14722 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.3:42415
I20260811 18:59:57.120332 14860 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.3:42415 every 8 connection(s)
I20260811 18:59:57.121356 14722 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
I20260811 18:59:57.122709 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 14722
I20260811 18:59:57.122936 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474792496492-13748-0/minicluster-data/ts-2/wal/instance
I20260811 18:59:57.123230 13748 external_mini_cluster.cc:1428] Setting key f7a275bad04cd856f7e585c447cf6767
Aug 11 18:59:57 dist-test-slave-0kls krb5kdc[14318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474797, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.3@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
I20260811 18:59:57.135764 14409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:57.123661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.3:46267 (local address 127.13.109.62:43179)
0811 18:59:57.123839 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:57.123844 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:57.124625 (+ 781us) server_negotiation.cc:407] Connection header received
0811 18:59:57.126145 (+ 1520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:57.126148 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:57.126196 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:57.126267 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:57.127437 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:57.127994 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:57.129154 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:57.129375 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:57.129455 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:57.132035 (+ 2580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:57.132039 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:57.132041 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:59:57.132069 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:57.133599 (+ 1530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:57.134197 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:57.134200 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:57.134202 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:57.134285 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:57.134603 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:57.134605 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:57.134606 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:57.134802 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:57.134878 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:57.135421 (+ 543us) server_negotiation.cc:299] Negotiation successful
0811 18:59:57.135594 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 18:59:57.136667 14863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:57.123949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:43179 (local address 127.13.109.3:46267)
0811 18:59:57.124463 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:57.124520 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 18:59:57.125931 (+ 1411us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:59:57.126439 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:59:57.126446 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:59:57.126819 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:59:57.127312 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:57.127326 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:57.128104 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:57.128106 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:57.129034 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:57.129044 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:57.129518 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:57.129520 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:57.129669 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:57.130329 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:59:57.130346 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:59:57.131847 (+ 1501us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:59:57.133713 (+ 1866us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:59:57.133718 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:59:57.133732 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:59:57.134088 (+ 356us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:59:57.134371 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:59:57.134373 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:59:57.134374 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:59:57.134517 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:59:57.134891 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:59:57.134897 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:59:57.135172 (+ 275us) client_negotiation.cc:769] Sending connection context
0811 18:59:57.135345 (+ 173us) client_negotiation.cc:240] Negotiation successful
0811 18:59:57.135637 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":146,"threads_started":1}
I20260811 18:59:57.137979 14861 heartbeater.cc:347] Connected to a master server at 127.13.109.62:43179
I20260811 18:59:57.138257 14861 heartbeater.cc:464] Registering TS with master...
I20260811 18:59:57.138850 14861 heartbeater.cc:511] Master 127.13.109.62:43179 requested a full tablet report, sending...
I20260811 18:59:57.140002 14366 ts_manager.cc:194] Registered new tserver with Master: 23adebc4e4204ca1978d788c9c5222b5 (127.13.109.3:42415)
I20260811 18:59:57.140636 14366 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.109.3@KRBTEST.COM'} at 127.13.109.3:46267
I20260811 18:59:57.150161 13748 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 18:59:57.163023 14409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:57.154968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52042 (local address 127.13.109.62:43179)
0811 18:59:57.155247 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:57.155253 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:57.155409 (+ 156us) server_negotiation.cc:407] Connection header received
0811 18:59:57.155601 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:57.155604 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:57.155649 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:57.155745 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:57.156532 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:57.157077 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:57.158015 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:57.158245 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:57.158363 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:57.159840 (+ 1477us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:57.159846 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:57.159854 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:59:57.159884 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:57.161107 (+ 1223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:57.161579 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:57.161583 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:57.161585 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:57.161675 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:57.161999 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:57.162003 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:57.162005 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:57.162258 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:57.162416 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:57.162680 (+ 264us) server_negotiation.cc:299] Negotiation successful
0811 18:59:57.162814 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175}
I20260811 18:59:57.169288 14366 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52042:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:59:57.172749 14366 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:59:57.189081 14872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:57.182685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.1:46169 (local address 127.0.0.1:36560)
0811 18:59:57.183400 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:57.183457 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 18:59:57.183597 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:59:57.184752 (+ 1155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:59:57.184754 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:59:57.184767 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:59:57.184875 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:57.184903 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:57.186167 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:57.186173 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:57.187466 (+ 1293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:57.187477 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:57.188448 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:57.188452 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:57.188583 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:57.188630 (+ 47us) client_negotiation.cc:769] Sending connection context
0811 18:59:57.188753 (+ 123us) client_negotiation.cc:240] Negotiation successful
0811 18:59:57.188911 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":618,"mutex_wait_us":122,"spinlock_wait_cycles":3200,"thread_start_us":131,"threads_started":1}
I20260811 18:59:57.189085 14875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:57.182864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36560 (local address 127.13.109.1:46169)
0811 18:59:57.184292 (+ 1428us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:57.184299 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:57.184321 (+ 22us) server_negotiation.cc:407] Connection header received
0811 18:59:57.184395 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:57.184409 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:57.184602 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:57.185115 (+ 513us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:59:57.185232 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:57.186013 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:57.187598 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:57.188309 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:57.188420 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:57.188582 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:57.188803 (+ 221us) server_negotiation.cc:299] Negotiation successful
0811 18:59:57.188923 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1330,"thread_start_us":124,"threads_started":1}
I20260811 18:59:57.189318 14873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:57.182504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.2:43341 (local address 127.0.0.1:40906)
0811 18:59:57.183400 (+ 896us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:57.183457 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 18:59:57.183597 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:59:57.184507 (+ 910us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:59:57.184515 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:59:57.184538 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:59:57.184704 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:57.184712 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:57.186297 (+ 1585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:57.186301 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:57.187924 (+ 1623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:57.187936 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:57.188936 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:57.188939 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:57.189031 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:57.189064 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 18:59:57.189117 (+ 53us) client_negotiation.cc:240] Negotiation successful
0811 18:59:57.189195 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":698,"mutex_wait_us":220,"spinlock_wait_cycles":7808,"thread_start_us":142,"threads_started":1}
I20260811 18:59:57.189615 14877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:57.183364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.3:42415 (local address 127.0.0.1:46060)
0811 18:59:57.183872 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:57.183885 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:59:57.183997 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:59:57.184712 (+ 715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:59:57.184716 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:59:57.184738 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:59:57.184881 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:57.184888 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:57.186304 (+ 1416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:57.186306 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:57.187928 (+ 1622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:57.187936 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:57.189191 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:57.189194 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:57.189323 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:57.189345 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 18:59:57.189413 (+ 68us) client_negotiation.cc:240] Negotiation successful
0811 18:59:57.189483 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":414,"mutex_wait_us":159,"thread_start_us":94,"threads_started":1}
I20260811 18:59:57.190090 14874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:57.182708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40906 (local address 127.13.109.2:43341)
0811 18:59:57.183936 (+ 1228us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:57.183949 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:57.184007 (+ 58us) server_negotiation.cc:407] Connection header received
0811 18:59:57.184091 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:57.184098 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:57.184328 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:57.184504 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:59:57.184846 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:57.186129 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:57.188082 (+ 1953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:57.188809 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:57.189599 (+ 790us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:57.189745 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:57.189798 (+ 53us) server_negotiation.cc:299] Negotiation successful
0811 18:59:57.189925 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1085,"thread_start_us":118,"threads_started":1}
I20260811 18:59:57.190166 14878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:57.183743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46060 (local address 127.13.109.3:42415)
0811 18:59:57.184200 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:57.184211 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:57.184275 (+ 64us) server_negotiation.cc:407] Connection header received
0811 18:59:57.184337 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:57.184341 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:57.184530 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:57.184693 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:59:57.185010 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:57.186172 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:57.188076 (+ 1904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:57.188878 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:57.188982 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:57.189121 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:57.189896 (+ 775us) server_negotiation.cc:299] Negotiation successful
0811 18:59:57.190015 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":155,"threads_started":1}
I20260811 18:59:57.192941 14648 tablet_service.cc:1467] Processing CreateTablet for tablet f3b4d6f543414f0eb03325190ed472fa (DEFAULT_TABLE table=test-table [id=eaf92fdbdc074ce080ed88bc6d4f657e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:59:57.193065 14504 tablet_service.cc:1467] Processing CreateTablet for tablet f3b4d6f543414f0eb03325190ed472fa (DEFAULT_TABLE table=test-table [id=eaf92fdbdc074ce080ed88bc6d4f657e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:59:57.193068 14791 tablet_service.cc:1467] Processing CreateTablet for tablet f3b4d6f543414f0eb03325190ed472fa (DEFAULT_TABLE table=test-table [id=eaf92fdbdc074ce080ed88bc6d4f657e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:59:57.194128 14648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f3b4d6f543414f0eb03325190ed472fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:59:57.194149 14504 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f3b4d6f543414f0eb03325190ed472fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:59:57.194149 14791 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f3b4d6f543414f0eb03325190ed472fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:59:57.201001 14879 tablet_bootstrap.cc:492] T f3b4d6f543414f0eb03325190ed472fa P 42cb7e3a86ca46e8ae5c70d28f1db3b6: Bootstrap starting.
I20260811 18:59:57.202350 14880 tablet_bootstrap.cc:492] T f3b4d6f543414f0eb03325190ed472fa P 24f54c31fefe4d938e63c41a6026ce55: Bootstrap starting.
I20260811 18:59:57.202786 14881 tablet_bootstrap.cc:492] T f3b4d6f543414f0eb03325190ed472fa P 23adebc4e4204ca1978d788c9c5222b5: Bootstrap starting.
I20260811 18:59:57.204413 14879 tablet_bootstrap.cc:654] T f3b4d6f543414f0eb03325190ed472fa P 42cb7e3a86ca46e8ae5c70d28f1db3b6: Neither blocks nor log segments found. Creating new log.
I20260811 18:59:57.204509 14880 tablet_bootstrap.cc:654] T f3b4d6f543414f0eb03325190ed472fa P 24f54c31fefe4d938e63c41a6026ce55: Neither blocks nor log segments found. Creating new log.
I20260811 18:59:57.204913 14881 tablet_bootstrap.cc:654] T f3b4d6f543414f0eb03325190ed472fa P 23adebc4e4204ca1978d788c9c5222b5: Neither blocks nor log segments found. Creating new log.
I20260811 18:59:57.205286 14880 log.cc:824] T f3b4d6f543414f0eb03325190ed472fa P 24f54c31fefe4d938e63c41a6026ce55: Log is configured to *not* fsync() on all Append() calls
I20260811 18:59:57.205369 14879 log.cc:824] T f3b4d6f543414f0eb03325190ed472fa P 42cb7e3a86ca46e8ae5c70d28f1db3b6: Log is configured to *not* fsync() on all Append() calls
I20260811 18:59:57.206144 14881 log.cc:824] T f3b4d6f543414f0eb03325190ed472fa P 23adebc4e4204ca1978d788c9c5222b5: Log is configured to *not* fsync() on all Append() calls
I20260811 18:59:57.207466 14880 tablet_bootstrap.cc:492] T f3b4d6f543414f0eb03325190ed472fa P 24f54c31fefe4d938e63c41a6026ce55: No bootstrap required, opened a new log
I20260811 18:59:57.207509 14879 tablet_bootstrap.cc:492] T f3b4d6f543414f0eb03325190ed472fa P 42cb7e3a86ca46e8ae5c70d28f1db3b6: No bootstrap required, opened a new log
I20260811 18:59:57.207693 14880 ts_tablet_manager.cc:1397] T f3b4d6f543414f0eb03325190ed472fa P 24f54c31fefe4d938e63c41a6026ce55: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 18:59:57.207695 14879 ts_tablet_manager.cc:1397] T f3b4d6f543414f0eb03325190ed472fa P 42cb7e3a86ca46e8ae5c70d28f1db3b6: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.002s
I20260811 18:59:57.208122 14881 tablet_bootstrap.cc:492] T f3b4d6f543414f0eb03325190ed472fa P 23adebc4e4204ca1978d788c9c5222b5: No bootstrap required, opened a new log
I20260811 18:59:57.208304 14881 ts_tablet_manager.cc:1397] T f3b4d6f543414f0eb03325190ed472fa P 23adebc4e4204ca1978d788c9c5222b5: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 18:59:57.211934 14880 raft_consensus.cc:348] T f3b4d6f543414f0eb03325190ed472fa P 24f54c31fefe4d938e63c41a6026ce55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42cb7e3a86ca46e8ae5c70d28f1db3b6" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 43341 } } peers { permanent_uuid: "23adebc4e4204ca1978d788c9c5222b5" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 42415 } } peers { permanent_uuid: "24f54c31fefe4d938e63c41a6026ce55" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 46169 } }
I20260811 18:59:57.211956 14879 raft_consensus.cc:348] T f3b4d6f543414f0eb03325190ed472fa P 42cb7e3a86ca46e8ae5c70d28f1db3b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42cb7e3a86ca46e8ae5c70d28f1db3b6" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 43341 } } peers { permanent_uuid: "23adebc4e4204ca1978d788c9c5222b5" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 42415 } } peers { permanent_uuid: "24f54c31fefe4d938e63c41a6026ce55" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 46169 } }
I20260811 18:59:57.212226 14880 raft_consensus.cc:374] T f3b4d6f543414f0eb03325190ed472fa P 24f54c31fefe4d938e63c41a6026ce55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:59:57.212272 14879 raft_consensus.cc:374] T f3b4d6f543414f0eb03325190ed472fa P 42cb7e3a86ca46e8ae5c70d28f1db3b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:59:57.212278 14880 raft_consensus.cc:729] T f3b4d6f543414f0eb03325190ed472fa P 24f54c31fefe4d938e63c41a6026ce55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24f54c31fefe4d938e63c41a6026ce55, State: Initialized, Role: FOLLOWER
I20260811 18:59:57.212337 14879 raft_consensus.cc:729] T f3b4d6f543414f0eb03325190ed472fa P 42cb7e3a86ca46e8ae5c70d28f1db3b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42cb7e3a86ca46e8ae5c70d28f1db3b6, State: Initialized, Role: FOLLOWER
I20260811 18:59:57.212177 14881 raft_consensus.cc:348] T f3b4d6f543414f0eb03325190ed472fa P 23adebc4e4204ca1978d788c9c5222b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42cb7e3a86ca46e8ae5c70d28f1db3b6" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 43341 } } peers { permanent_uuid: "23adebc4e4204ca1978d788c9c5222b5" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 42415 } } peers { permanent_uuid: "24f54c31fefe4d938e63c41a6026ce55" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 46169 } }
I20260811 18:59:57.212438 14881 raft_consensus.cc:374] T f3b4d6f543414f0eb03325190ed472fa P 23adebc4e4204ca1978d788c9c5222b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:59:57.212493 14881 raft_consensus.cc:729] T f3b4d6f543414f0eb03325190ed472fa P 23adebc4e4204ca1978d788c9c5222b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23adebc4e4204ca1978d788c9c5222b5, State: Initialized, Role: FOLLOWER
I20260811 18:59:57.212837 14880 consensus_queue.cc:260] T f3b4d6f543414f0eb03325190ed472fa P 24f54c31fefe4d938e63c41a6026ce55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42cb7e3a86ca46e8ae5c70d28f1db3b6" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 43341 } } peers { permanent_uuid: "23adebc4e4204ca1978d788c9c5222b5" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 42415 } } peers { permanent_uuid: "24f54c31fefe4d938e63c41a6026ce55" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 46169 } }
I20260811 18:59:57.212857 14881 consensus_queue.cc:260] T f3b4d6f543414f0eb03325190ed472fa P 23adebc4e4204ca1978d788c9c5222b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42cb7e3a86ca46e8ae5c70d28f1db3b6" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 43341 } } peers { permanent_uuid: "23adebc4e4204ca1978d788c9c5222b5" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 42415 } } peers { permanent_uuid: "24f54c31fefe4d938e63c41a6026ce55" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 46169 } }
I20260811 18:59:57.212838 14879 consensus_queue.cc:260] T f3b4d6f543414f0eb03325190ed472fa P 42cb7e3a86ca46e8ae5c70d28f1db3b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42cb7e3a86ca46e8ae5c70d28f1db3b6" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 43341 } } peers { permanent_uuid: "23adebc4e4204ca1978d788c9c5222b5" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 42415 } } peers { permanent_uuid: "24f54c31fefe4d938e63c41a6026ce55" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 46169 } }
I20260811 18:59:57.213626 14576 heartbeater.cc:503] Master 127.13.109.62:43179 was elected leader, sending a full tablet report...
I20260811 18:59:57.214028 14879 ts_tablet_manager.cc:1428] T f3b4d6f543414f0eb03325190ed472fa P 42cb7e3a86ca46e8ae5c70d28f1db3b6: Time spent starting tablet: real 0.006s user 0.006s sys 0.002s
I20260811 18:59:57.214110 14716 heartbeater.cc:503] Master 127.13.109.62:43179 was elected leader, sending a full tablet report...
I20260811 18:59:57.214263 14861 heartbeater.cc:503] Master 127.13.109.62:43179 was elected leader, sending a full tablet report...
I20260811 18:59:57.214720 14881 ts_tablet_manager.cc:1428] T f3b4d6f543414f0eb03325190ed472fa P 23adebc4e4204ca1978d788c9c5222b5: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 18:59:57.215193 14880 ts_tablet_manager.cc:1428] T f3b4d6f543414f0eb03325190ed472fa P 24f54c31fefe4d938e63c41a6026ce55: Time spent starting tablet: real 0.007s user 0.001s sys 0.006s
I20260811 18:59:57.246495 14887 raft_consensus.cc:482] T f3b4d6f543414f0eb03325190ed472fa P 42cb7e3a86ca46e8ae5c70d28f1db3b6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:59:57.246781 14887 raft_consensus.cc:504] T f3b4d6f543414f0eb03325190ed472fa P 42cb7e3a86ca46e8ae5c70d28f1db3b6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42cb7e3a86ca46e8ae5c70d28f1db3b6" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 43341 } } peers { permanent_uuid: "23adebc4e4204ca1978d788c9c5222b5" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 42415 } } peers { permanent_uuid: "24f54c31fefe4d938e63c41a6026ce55" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 46169 } }
I20260811 18:59:57.248140 14887 leader_election.cc:288] T f3b4d6f543414f0eb03325190ed472fa P 42cb7e3a86ca46e8ae5c70d28f1db3b6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 23adebc4e4204ca1978d788c9c5222b5 (127.13.109.3:42415), 24f54c31fefe4d938e63c41a6026ce55 (127.13.109.1:46169)
I20260811 18:59:57.253225 14878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:57.248604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.2:39311 (local address 127.13.109.3:42415)
0811 18:59:57.248765 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:57.248772 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:57.248788 (+ 16us) server_negotiation.cc:407] Connection header received
0811 18:59:57.248850 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:57.248853 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:57.248906 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:57.248981 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:59:57.249330 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:57.250158 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:57.251849 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:57.252513 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:57.252646 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:57.252705 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:57.253005 (+ 300us) server_negotiation.cc:299] Negotiation successful
0811 18:59:57.253083 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 18:59:57.253225 14875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:57.248909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.2:45353 (local address 127.13.109.1:46169)
0811 18:59:57.249073 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:57.249077 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:57.249091 (+ 14us) server_negotiation.cc:407] Connection header received
0811 18:59:57.249142 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:57.249144 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:57.249214 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:57.249300 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:59:57.249644 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:57.250282 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:57.251849 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:57.252512 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:57.252646 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:57.252710 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:57.253005 (+ 295us) server_negotiation.cc:299] Negotiation successful
0811 18:59:57.253083 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 18:59:57.253232 14718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:57.248336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.3:42415 (local address 127.13.109.2:39311)
0811 18:59:57.248545 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:57.248572 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 18:59:57.248674 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:59:57.249055 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:59:57.249057 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:59:57.249074 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:59:57.249203 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:57.249211 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:57.250279 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:57.250285 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:57.251715 (+ 1430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:57.251720 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:57.252658 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:57.252662 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:57.252834 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:57.252875 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 18:59:57.252960 (+ 85us) client_negotiation.cc:240] Negotiation successful
0811 18:59:57.253104 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"mutex_wait_us":180}
I20260811 18:59:57.253244 14889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:57.248463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.1:46169 (local address 127.13.109.2:45353)
0811 18:59:57.248822 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:57.248834 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 18:59:57.248922 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:59:57.249348 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:59:57.249351 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:59:57.249372 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:59:57.249502 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:57.249508 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:57.250364 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:57.250367 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:57.251707 (+ 1340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:57.251720 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:57.252658 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:57.252662 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:57.252823 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:57.252884 (+ 61us) client_negotiation.cc:769] Sending connection context
0811 18:59:57.252960 (+ 76us) client_negotiation.cc:240] Negotiation successful
0811 18:59:57.253068 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"mutex_wait_us":119,"thread_start_us":143,"threads_started":1}
I20260811 18:59:57.254328 14525 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3b4d6f543414f0eb03325190ed472fa" candidate_uuid: "42cb7e3a86ca46e8ae5c70d28f1db3b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24f54c31fefe4d938e63c41a6026ce55" is_pre_election: true
I20260811 18:59:57.254328 14811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3b4d6f543414f0eb03325190ed472fa" candidate_uuid: "42cb7e3a86ca46e8ae5c70d28f1db3b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23adebc4e4204ca1978d788c9c5222b5" is_pre_election: true
I20260811 18:59:57.254716 14811 raft_consensus.cc:2454] T f3b4d6f543414f0eb03325190ed472fa P 23adebc4e4204ca1978d788c9c5222b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 42cb7e3a86ca46e8ae5c70d28f1db3b6 in term 0.
I20260811 18:59:57.254716 14525 raft_consensus.cc:2454] T f3b4d6f543414f0eb03325190ed472fa P 24f54c31fefe4d938e63c41a6026ce55 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 42cb7e3a86ca46e8ae5c70d28f1db3b6 in term 0.
I20260811 18:59:57.255323 14601 leader_election.cc:302] T f3b4d6f543414f0eb03325190ed472fa P 42cb7e3a86ca46e8ae5c70d28f1db3b6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 23adebc4e4204ca1978d788c9c5222b5, 24f54c31fefe4d938e63c41a6026ce55, 42cb7e3a86ca46e8ae5c70d28f1db3b6; no voters:
I20260811 18:59:57.255692 14887 raft_consensus.cc:2781] T f3b4d6f543414f0eb03325190ed472fa P 42cb7e3a86ca46e8ae5c70d28f1db3b6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:59:57.255780 14887 raft_consensus.cc:482] T f3b4d6f543414f0eb03325190ed472fa P 42cb7e3a86ca46e8ae5c70d28f1db3b6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:59:57.255812 14887 raft_consensus.cc:3037] T f3b4d6f543414f0eb03325190ed472fa P 42cb7e3a86ca46e8ae5c70d28f1db3b6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:59:57.256935 14887 raft_consensus.cc:504] T f3b4d6f543414f0eb03325190ed472fa P 42cb7e3a86ca46e8ae5c70d28f1db3b6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42cb7e3a86ca46e8ae5c70d28f1db3b6" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 43341 } } peers { permanent_uuid: "23adebc4e4204ca1978d788c9c5222b5" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 42415 } } peers { permanent_uuid: "24f54c31fefe4d938e63c41a6026ce55" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 46169 } }
I20260811 18:59:57.257339 14887 leader_election.cc:288] T f3b4d6f543414f0eb03325190ed472fa P 42cb7e3a86ca46e8ae5c70d28f1db3b6 [CANDIDATE]: Term 1 election: Requested vote from peers 23adebc4e4204ca1978d788c9c5222b5 (127.13.109.3:42415), 24f54c31fefe4d938e63c41a6026ce55 (127.13.109.1:46169)
I20260811 18:59:57.257865 14525 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3b4d6f543414f0eb03325190ed472fa" candidate_uuid: "42cb7e3a86ca46e8ae5c70d28f1db3b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24f54c31fefe4d938e63c41a6026ce55"
I20260811 18:59:57.257864 14811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3b4d6f543414f0eb03325190ed472fa" candidate_uuid: "42cb7e3a86ca46e8ae5c70d28f1db3b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23adebc4e4204ca1978d788c9c5222b5"
I20260811 18:59:57.258005 14811 raft_consensus.cc:3037] T f3b4d6f543414f0eb03325190ed472fa P 23adebc4e4204ca1978d788c9c5222b5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:59:57.258005 14525 raft_consensus.cc:3037] T f3b4d6f543414f0eb03325190ed472fa P 24f54c31fefe4d938e63c41a6026ce55 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:59:57.259430 14811 raft_consensus.cc:2454] T f3b4d6f543414f0eb03325190ed472fa P 23adebc4e4204ca1978d788c9c5222b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 42cb7e3a86ca46e8ae5c70d28f1db3b6 in term 1.
I20260811 18:59:57.259430 14525 raft_consensus.cc:2454] T f3b4d6f543414f0eb03325190ed472fa P 24f54c31fefe4d938e63c41a6026ce55 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 42cb7e3a86ca46e8ae5c70d28f1db3b6 in term 1.
I20260811 18:59:57.259799 14601 leader_election.cc:302] T f3b4d6f543414f0eb03325190ed472fa P 42cb7e3a86ca46e8ae5c70d28f1db3b6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23adebc4e4204ca1978d788c9c5222b5, 42cb7e3a86ca46e8ae5c70d28f1db3b6; no voters:
I20260811 18:59:57.260020 14887 raft_consensus.cc:2781] T f3b4d6f543414f0eb03325190ed472fa P 42cb7e3a86ca46e8ae5c70d28f1db3b6 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:59:57.260303 14887 raft_consensus.cc:686] T f3b4d6f543414f0eb03325190ed472fa P 42cb7e3a86ca46e8ae5c70d28f1db3b6 [term 1 LEADER]: Becoming Leader. State: Replica: 42cb7e3a86ca46e8ae5c70d28f1db3b6, State: Running, Role: LEADER
I20260811 18:59:57.260653 14887 consensus_queue.cc:237] T f3b4d6f543414f0eb03325190ed472fa P 42cb7e3a86ca46e8ae5c70d28f1db3b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42cb7e3a86ca46e8ae5c70d28f1db3b6" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 43341 } } peers { permanent_uuid: "23adebc4e4204ca1978d788c9c5222b5" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 42415 } } peers { permanent_uuid: "24f54c31fefe4d938e63c41a6026ce55" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 46169 } }
I20260811 18:59:57.263890 14367 catalog_manager.cc:5495] T f3b4d6f543414f0eb03325190ed472fa P 42cb7e3a86ca46e8ae5c70d28f1db3b6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 42cb7e3a86ca46e8ae5c70d28f1db3b6 (127.13.109.2). New cstate: current_term: 1 leader_uuid: "42cb7e3a86ca46e8ae5c70d28f1db3b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42cb7e3a86ca46e8ae5c70d28f1db3b6" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 43341 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "23adebc4e4204ca1978d788c9c5222b5" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 42415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24f54c31fefe4d938e63c41a6026ce55" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 46169 } health_report { overall_health: UNKNOWN } } }
Aug 11 18:59:57 dist-test-slave-0kls krb5kdc[14318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474796, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.109.1@KRBTEST.COM
I20260811 18:59:57.296607 14875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:57.287073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36574 (local address 127.13.109.1:46169)
0811 18:59:57.287232 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:57.287236 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:57.287418 (+ 182us) server_negotiation.cc:407] Connection header received
0811 18:59:57.287526 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:57.287529 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:57.287589 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:57.287667 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:57.288353 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:57.288874 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:57.289703 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:57.289953 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:57.290045 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:57.292790 (+ 2745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:57.292804 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:57.292822 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:59:57.292851 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:57.294694 (+ 1843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:57.295171 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:57.295177 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:57.295178 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:57.295278 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:57.295592 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:57.295596 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:57.295598 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:57.295877 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:57.296130 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:57.296299 (+ 169us) server_negotiation.cc:299] Negotiation successful
0811 18:59:57.296428 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
Aug 11 18:59:57 dist-test-slave-0kls krb5kdc[14318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474797, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 18:59:57 dist-test-slave-0kls krb5kdc[14318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474797, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.13.109.1@KRBTEST.COM
I20260811 18:59:57.326380 14875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:57.317101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36584 (local address 127.13.109.1:46169)
0811 18:59:57.317242 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:57.317246 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:57.317420 (+ 174us) server_negotiation.cc:407] Connection header received
0811 18:59:57.317557 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:57.317560 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:57.317606 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:57.317693 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:57.318592 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:57.319191 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:57.320032 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:57.320255 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:57.320341 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:57.323275 (+ 2934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:57.323279 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:57.323282 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:59:57.323320 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:57.324523 (+ 1203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:57.324969 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:57.324973 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:57.324975 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:57.325068 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:57.325364 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:57.325367 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:57.325369 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:57.325567 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:57.325711 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:57.326081 (+ 370us) server_negotiation.cc:299] Negotiation successful
0811 18:59:57.326213 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
Aug 11 18:59:57 dist-test-slave-0kls krb5kdc[14318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474797, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 18:59:57 dist-test-slave-0kls krb5kdc[14318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474797, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.109.1@KRBTEST.COM
I20260811 18:59:57.354118 14875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:57.345460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36600 (local address 127.13.109.1:46169)
0811 18:59:57.345601 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:57.345606 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:57.345829 (+ 223us) server_negotiation.cc:407] Connection header received
0811 18:59:57.346003 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:57.346006 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:57.346053 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:57.346153 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:57.346916 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:57.347435 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:57.348293 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:57.348517 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:57.348627 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:57.351124 (+ 2497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:57.351129 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:57.351134 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:59:57.351169 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:57.352681 (+ 1512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:57.353051 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:57.353053 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:57.353054 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:57.353122 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:57.353377 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:57.353379 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:57.353380 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:57.353536 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:57.353633 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:57.353894 (+ 261us) server_negotiation.cc:299] Negotiation successful
0811 18:59:57.353992 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
W20260811 18:59:57.355571 14482 tablet_service.cc:2908] Rejecting scan request for tablet f3b4d6f543414f0eb03325190ed472fa: Uninitialized: clean time has not yet been initialized
I20260811 18:59:57.358224 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 14434
I20260811 18:59:57.369733 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 14581
W20260811 18:59:57.372561 14862 tablet.cc:2369] T f3b4d6f543414f0eb03325190ed472fa P 23adebc4e4204ca1978d788c9c5222b5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:59:57.380030 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 14722
I20260811 18:59:57.388062 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 14333
2026-08-11T18:59:57Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2475 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:59:57 dist-test-slave-0kls krb5kdc[14905](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:59:57 dist-test-slave-0kls krb5kdc[14905](info): set up 2 sockets
Aug 11 18:59:57 dist-test-slave-0kls krb5kdc[14905](info): commencing operation
krb5kdc: starting...
W20260811 18:59:58.489959 13748 mini_kdc.cc:121] Time spent starting KDC: real 1.076s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/krb5kdc/test-user.keytab.
Aug 11 18:59:58 dist-test-slave-0kls krb5kdc[14905](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474798, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:59:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:59:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.109.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.109.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.109.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab.
Entry for principal kudu/127.13.109.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab.
I20260811 18:59:58.622926 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.62:43061
--webserver_interface=127.13.109.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab
--principal=kudu/127.13.109.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:44071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.109.62:43061
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:59:58.747061 14918 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:59:58.747429 14918 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:59:58.747510 14918 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:59:58.752182 14918 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:59:58.752293 14918 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 18:59:58.752326 14918 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:59:58.752368 14918 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:59:58.752405 14918 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:59:58.752440 14918 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:59:58.758226 14918 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:44071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.109.62:43061
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab
--principal=kudu/127.13.109.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.62:43061
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.109.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14918
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:59:58.759449 14918 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:59:58.760571 14918 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:59:58.769577 14928 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:58.769585 14925 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:58.769593 14924 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:59:58.769846 14918 server_base.cc:1034] running on GCE node
I20260811 18:59:58.770767 14918 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:59:58.771925 14918 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:59:58.773123 14918 hybrid_clock.cc:648] HybridClock initialized: now 1786474798773114 us; error 31 us; skew 500 ppm
Aug 11 18:59:58 dist-test-slave-0kls krb5kdc[14905](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474798, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:59:58.779994 14918 init.cc:377] Logged in from keytab as kudu/127.13.109.62@KRBTEST.COM (short username kudu)
I20260811 18:59:58.781023 14918 webserver.cc:460] Webserver started at http://127.13.109.62:44193/ using document root <none> and password file <none>
I20260811 18:59:58.781581 14918 fs_manager.cc:359] Metadata directory not provided
I20260811 18:59:58.781641 14918 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:59:58.781881 14918 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:59:58.783938 14918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/minicluster-data/master-0/data/instance:
uuid: "1f76e5475fab4ce384e2d95956b9aa32"
format_stamp: "Formatted at 2026-08-11 18:59:58 on dist-test-slave-0kls"
server_key: "54e97c05d4ef832aec591e39e03c4204"
server_key_iv: "f6926b1e747741c2fa8f35446d9bb529"
server_key_version: "encryptionkey@0"
I20260811 18:59:58.784463 14918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/minicluster-data/master-0/wal/instance:
uuid: "1f76e5475fab4ce384e2d95956b9aa32"
format_stamp: "Formatted at 2026-08-11 18:59:58 on dist-test-slave-0kls"
server_key: "54e97c05d4ef832aec591e39e03c4204"
server_key_iv: "f6926b1e747741c2fa8f35446d9bb529"
server_key_version: "encryptionkey@0"
I20260811 18:59:58.788269 14918 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.005s
I20260811 18:59:58.790719 14935 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:58.791934 14918 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 18:59:58.792078 14918 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/minicluster-data/master-0/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/minicluster-data/master-0/wal
uuid: "1f76e5475fab4ce384e2d95956b9aa32"
format_stamp: "Formatted at 2026-08-11 18:59:58 on dist-test-slave-0kls"
server_key: "54e97c05d4ef832aec591e39e03c4204"
server_key_iv: "f6926b1e747741c2fa8f35446d9bb529"
server_key_version: "encryptionkey@0"
I20260811 18:59:58.792199 14918 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:59:58.803565 14918 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:59:58.806648 14918 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:59:58.806865 14918 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:59:58.815785 14918 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.62:43061
I20260811 18:59:58.815809 15001 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.62:43061 every 8 connection(s)
I20260811 18:59:58.816753 14918 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
I20260811 18:59:58.819624 15002 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:59:58.821455 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 14918
I20260811 18:59:58.821580 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/minicluster-data/master-0/wal/instance
I20260811 18:59:58.821883 13748 external_mini_cluster.cc:1428] Setting key 7ec3562ffec5a900c6733413ca16682e
I20260811 18:59:58.826196 15002 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f76e5475fab4ce384e2d95956b9aa32: Bootstrap starting.
I20260811 18:59:58.828603 15002 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1f76e5475fab4ce384e2d95956b9aa32: Neither blocks nor log segments found. Creating new log.
I20260811 18:59:58.829452 15002 log.cc:824] T 00000000000000000000000000000000 P 1f76e5475fab4ce384e2d95956b9aa32: Log is configured to *not* fsync() on all Append() calls
Aug 11 18:59:58 dist-test-slave-0kls krb5kdc[14905](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474798, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
I20260811 18:59:58.831934 15002 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f76e5475fab4ce384e2d95956b9aa32: No bootstrap required, opened a new log
I20260811 18:59:58.835407 15002 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1f76e5475fab4ce384e2d95956b9aa32 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f76e5475fab4ce384e2d95956b9aa32" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 43061 } }
I20260811 18:59:58.835678 15002 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1f76e5475fab4ce384e2d95956b9aa32 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:59:58.835770 15002 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1f76e5475fab4ce384e2d95956b9aa32 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f76e5475fab4ce384e2d95956b9aa32, State: Initialized, Role: FOLLOWER
I20260811 18:59:58.836292 15002 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1f76e5475fab4ce384e2d95956b9aa32 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f76e5475fab4ce384e2d95956b9aa32" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 43061 } }
I20260811 18:59:58.836459 15002 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1f76e5475fab4ce384e2d95956b9aa32 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:59:58.836544 15002 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1f76e5475fab4ce384e2d95956b9aa32 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:59:58.836668 15002 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1f76e5475fab4ce384e2d95956b9aa32 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:59:58.837744 15005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:58.823466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41604 (local address 127.13.109.62:43061)
0811 18:59:58.823915 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:58.823931 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:58.823968 (+ 37us) server_negotiation.cc:407] Connection header received
0811 18:59:58.824602 (+ 634us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:58.824618 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:58.824946 (+ 328us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:58.825282 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:58.826138 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:58.827351 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:58.828219 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:58.828763 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:58.828947 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:58.832040 (+ 3093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:58.832046 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:58.832066 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:59:58.832095 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:58.834542 (+ 2447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:58.835056 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:58.835062 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:58.835066 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:58.835183 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:58.835492 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:58.835496 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:58.835498 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:58.836049 (+ 551us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:58.836263 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:58.836558 (+ 295us) server_negotiation.cc:299] Negotiation successful
0811 18:59:58.836883 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":132,"threads_started":1}
I20260811 18:59:58.837759 15002 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1f76e5475fab4ce384e2d95956b9aa32 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f76e5475fab4ce384e2d95956b9aa32" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 43061 } }
I20260811 18:59:58.838275 15002 leader_election.cc:302] T 00000000000000000000000000000000 P 1f76e5475fab4ce384e2d95956b9aa32 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1f76e5475fab4ce384e2d95956b9aa32; no voters:
I20260811 18:59:58.838676 15002 leader_election.cc:288] T 00000000000000000000000000000000 P 1f76e5475fab4ce384e2d95956b9aa32 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:59:58.838860 15007 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1f76e5475fab4ce384e2d95956b9aa32 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:59:58.839172 15007 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1f76e5475fab4ce384e2d95956b9aa32 [term 1 LEADER]: Becoming Leader. State: Replica: 1f76e5475fab4ce384e2d95956b9aa32, State: Running, Role: LEADER
I20260811 18:59:58.839538 15007 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1f76e5475fab4ce384e2d95956b9aa32 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f76e5475fab4ce384e2d95956b9aa32" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 43061 } }
I20260811 18:59:58.840133 15002 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1f76e5475fab4ce384e2d95956b9aa32 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:59:58.841521 15009 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f76e5475fab4ce384e2d95956b9aa32 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1f76e5475fab4ce384e2d95956b9aa32. Latest consensus state: current_term: 1 leader_uuid: "1f76e5475fab4ce384e2d95956b9aa32" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f76e5475fab4ce384e2d95956b9aa32" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 43061 } } }
I20260811 18:59:58.841677 15009 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f76e5475fab4ce384e2d95956b9aa32 [sys.catalog]: This master's current role is: LEADER
I20260811 18:59:58.842007 15008 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f76e5475fab4ce384e2d95956b9aa32 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1f76e5475fab4ce384e2d95956b9aa32" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f76e5475fab4ce384e2d95956b9aa32" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 43061 } } }
I20260811 18:59:58.842114 15016 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:59:58.842139 15008 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f76e5475fab4ce384e2d95956b9aa32 [sys.catalog]: This master's current role is: LEADER
I20260811 18:59:58.845444 15016 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:59:58.851472 15016 catalog_manager.cc:1347] Generated new cluster ID: 423a6f004890485d8a788963421d4d8d
I20260811 18:59:58.851544 15016 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:59:58.860551 15016 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:59:58.861887 15016 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:59:58.870090 15016 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1f76e5475fab4ce384e2d95956b9aa32: Generated new TSK 0
I20260811 18:59:58.870955 15016 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.109.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.109.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.109.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.1.keytab.
Entry for principal kudu/127.13.109.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.1.keytab.
I20260811 18:59:58.909260 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.1:0
--local_ip_for_outbound_sockets=127.13.109.1
--webserver_interface=127.13.109.1
--webserver_port=0
--tserver_master_addrs=127.13.109.62:43061
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.1.keytab
--principal=kudu/127.13.109.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:44071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:59:59.031826 15029 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:59:59.032226 15029 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:59:59.032321 15029 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:59:59.036651 15029 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:59:59.036800 15029 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:59:59.036878 15029 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 18:59:59.036978 15029 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.1
I20260811 18:59:59.043391 15029 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:44071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.1.keytab
--principal=kudu/127.13.109.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.109.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.109.62:43061
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15029
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.1
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:59:59.044811 15029 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:59:59.045997 15029 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:59:59.056896 15035 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:59:59.056955 15029 server_base.cc:1034] running on GCE node
W20260811 18:59:59.056900 15036 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:59.056896 15038 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:59:59.057655 15029 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:59:59.058444 15029 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:59:59.059652 15029 hybrid_clock.cc:648] HybridClock initialized: now 1786474799059626 us; error 48 us; skew 500 ppm
Aug 11 18:59:59 dist-test-slave-0kls krb5kdc[14905](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474799, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:59:59.062716 15029 init.cc:377] Logged in from keytab as kudu/127.13.109.1@KRBTEST.COM (short username kudu)
I20260811 18:59:59.063702 15029 webserver.cc:460] Webserver started at http://127.13.109.1:35641/ using document root <none> and password file <none>
I20260811 18:59:59.064317 15029 fs_manager.cc:359] Metadata directory not provided
I20260811 18:59:59.064414 15029 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:59:59.064685 15029 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:59:59.066982 15029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/minicluster-data/ts-0/data/instance:
uuid: "bbfd45351bfd41879acc3a7d6073fc55"
format_stamp: "Formatted at 2026-08-11 18:59:59 on dist-test-slave-0kls"
server_key: "12ac1b59ff0d23c6401a6cd931c1a645"
server_key_iv: "63688474b4a6bda73afb1ff815be42d0"
server_key_version: "encryptionkey@0"
I20260811 18:59:59.067658 15029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/minicluster-data/ts-0/wal/instance:
uuid: "bbfd45351bfd41879acc3a7d6073fc55"
format_stamp: "Formatted at 2026-08-11 18:59:59 on dist-test-slave-0kls"
server_key: "12ac1b59ff0d23c6401a6cd931c1a645"
server_key_iv: "63688474b4a6bda73afb1ff815be42d0"
server_key_version: "encryptionkey@0"
I20260811 18:59:59.071830 15029 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 18:59:59.074584 15046 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:59.075896 15029 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 18:59:59.076056 15029 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/minicluster-data/ts-0/wal
uuid: "bbfd45351bfd41879acc3a7d6073fc55"
format_stamp: "Formatted at 2026-08-11 18:59:59 on dist-test-slave-0kls"
server_key: "12ac1b59ff0d23c6401a6cd931c1a645"
server_key_iv: "63688474b4a6bda73afb1ff815be42d0"
server_key_version: "encryptionkey@0"
I20260811 18:59:59.076185 15029 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:59:59.101670 15029 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:59:59.105427 15029 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:59:59.105674 15029 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:59:59.107334 15029 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:59:59.107421 15029 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:59.107498 15029 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:59:59.107558 15029 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 18:59:59 dist-test-slave-0kls krb5kdc[14905](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474799, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.1@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
I20260811 18:59:59.119964 15029 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.1:38161
I20260811 18:59:59.120041 15167 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.1:38161 every 8 connection(s)
I20260811 18:59:59.121251 15029 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
I20260811 18:59:59.123072 15005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:59.108417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.1:43769 (local address 127.13.109.62:43061)
0811 18:59:59.108559 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:59.108563 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:59.109891 (+ 1328us) server_negotiation.cc:407] Connection header received
0811 18:59:59.111140 (+ 1249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:59.111144 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:59.111241 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:59.111344 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:59.113213 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:59.113965 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:59.115436 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:59.115692 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:59.115822 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:59.118631 (+ 2809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:59.118642 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:59.118645 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:59:59.118675 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:59.120516 (+ 1841us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:59.121077 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:59.121081 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:59.121083 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:59.121200 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:59.121634 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:59.121640 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:59.121644 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:59.121978 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:59.122090 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:59.122731 (+ 641us) server_negotiation.cc:299] Negotiation successful
0811 18:59:59.122892 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 18:59:59.123849 15055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:59.109009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:43061 (local address 127.13.109.1:43769)
0811 18:59:59.109658 (+ 649us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:59.109723 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0811 18:59:59.110876 (+ 1153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:59:59.111601 (+ 725us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:59:59.111613 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:59:59.112209 (+ 596us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:59:59.112998 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:59.113026 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:59.114153 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:59.114156 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:59.115279 (+ 1123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:59.115290 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:59.115885 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:59.115890 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:59.116093 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:59.116746 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:59:59.116763 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:59:59.118431 (+ 1668us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:59:59.120653 (+ 2222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:59:59.120658 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:59:59.120672 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:59:59.120952 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:59:59.121332 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:59:59.121336 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:59:59.121337 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:59:59.121513 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:59:59.122098 (+ 585us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:59:59.122106 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:59:59.122451 (+ 345us) client_negotiation.cc:769] Sending connection context
0811 18:59:59.122678 (+ 227us) client_negotiation.cc:240] Negotiation successful
0811 18:59:59.123020 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":145,"threads_started":1}
I20260811 18:59:59.125082 15168 heartbeater.cc:347] Connected to a master server at 127.13.109.62:43061
I20260811 18:59:59.125254 15168 heartbeater.cc:464] Registering TS with master...
I20260811 18:59:59.125873 15168 heartbeater.cc:511] Master 127.13.109.62:43061 requested a full tablet report, sending...
I20260811 18:59:59.127619 14952 ts_manager.cc:194] Registered new tserver with Master: bbfd45351bfd41879acc3a7d6073fc55 (127.13.109.1:38161)
I20260811 18:59:59.128388 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 15029
I20260811 18:59:59.128518 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/minicluster-data/ts-0/wal/instance
I20260811 18:59:59.128868 13748 external_mini_cluster.cc:1428] Setting key 38863173d52709ec6a3046f31beb8c6f
I20260811 18:59:59.129149 14952 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.109.1@KRBTEST.COM'} at 127.13.109.1:43769
I20260811 18:59:59.135435 15005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:59.127686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.1:37981 (local address 127.13.109.62:43061)
0811 18:59:59.127812 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:59.127817 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:59.128041 (+ 224us) server_negotiation.cc:407] Connection header received
0811 18:59:59.128193 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:59.128196 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:59.128240 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:59.128326 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:59.128876 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:59.129593 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:59.130640 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:59.130887 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:59.131001 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:59.132180 (+ 1179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:59.132184 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:59.132187 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:59:59.132211 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:59.133704 (+ 1493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:59.134172 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:59.134177 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:59.134184 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:59.134268 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:59.134560 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:59.134567 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:59.134570 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:59.134756 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:59.134868 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:59.135142 (+ 274us) server_negotiation.cc:299] Negotiation successful
0811 18:59:59.135273 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 18:59:59.135444 15175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:59.127619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:43061 (local address 127.13.109.1:37981)
0811 18:59:59.127953 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:59.127966 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:59:59.128060 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:59:59.128367 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:59:59.128371 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:59:59.128602 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:59:59.128746 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:59.128753 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:59.129764 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:59.129769 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:59.130499 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:59.130509 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:59.131019 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:59.131023 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:59.131145 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:59.131679 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:59:59.131692 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:59:59.132036 (+ 344us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:59:59.133845 (+ 1809us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:59:59.133849 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:59:59.133852 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:59:59.134068 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:59:59.134373 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:59:59.134376 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:59:59.134378 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:59:59.134467 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:59:59.134858 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:59:59.134861 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:59:59.134965 (+ 104us) client_negotiation.cc:769] Sending connection context
0811 18:59:59.135075 (+ 110us) client_negotiation.cc:240] Negotiation successful
0811 18:59:59.135232 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"mutex_wait_us":4,"thread_start_us":112,"threads_started":1}
WARNING: no policy specified for kudu/127.13.109.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.109.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.109.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.2.keytab.
Entry for principal kudu/127.13.109.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.2.keytab.
I20260811 18:59:59.160497 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.2:0
--local_ip_for_outbound_sockets=127.13.109.2
--webserver_interface=127.13.109.2
--webserver_port=0
--tserver_master_addrs=127.13.109.62:43061
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.2.keytab
--principal=kudu/127.13.109.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:44071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:59:59.278278 15179 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:59:59.278590 15179 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:59:59.278640 15179 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:59:59.283229 15179 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:59:59.283311 15179 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:59:59.283356 15179 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 18:59:59.283390 15179 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.2
I20260811 18:59:59.289131 15179 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:44071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.2.keytab
--principal=kudu/127.13.109.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.109.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.109.62:43061
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15179
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.2
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:59:59.290398 15179 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:59:59.291445 15179 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:59:59.301990 15185 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:59.302030 15184 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:59.302022 15189 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:59:59.302237 15179 server_base.cc:1034] running on GCE node
I20260811 18:59:59.302636 15179 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:59:59.303303 15179 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:59:59.304493 15179 hybrid_clock.cc:648] HybridClock initialized: now 1786474799304459 us; error 46 us; skew 500 ppm
Aug 11 18:59:59 dist-test-slave-0kls krb5kdc[14905](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474799, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:59:59.307482 15179 init.cc:377] Logged in from keytab as kudu/127.13.109.2@KRBTEST.COM (short username kudu)
I20260811 18:59:59.308480 15179 webserver.cc:460] Webserver started at http://127.13.109.2:41527/ using document root <none> and password file <none>
I20260811 18:59:59.309082 15179 fs_manager.cc:359] Metadata directory not provided
I20260811 18:59:59.309201 15179 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:59:59.309449 15179 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:59:59.311691 15179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/minicluster-data/ts-1/data/instance:
uuid: "3cf97c97242e411a90d80ef88e985a9a"
format_stamp: "Formatted at 2026-08-11 18:59:59 on dist-test-slave-0kls"
server_key: "c3da689303318bddbc5fd4bf4a34e860"
server_key_iv: "5e0ff792b9ef7bce4a7ca5fe02661d32"
server_key_version: "encryptionkey@0"
I20260811 18:59:59.312315 15179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/minicluster-data/ts-1/wal/instance:
uuid: "3cf97c97242e411a90d80ef88e985a9a"
format_stamp: "Formatted at 2026-08-11 18:59:59 on dist-test-slave-0kls"
server_key: "c3da689303318bddbc5fd4bf4a34e860"
server_key_iv: "5e0ff792b9ef7bce4a7ca5fe02661d32"
server_key_version: "encryptionkey@0"
I20260811 18:59:59.316496 15179 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 18:59:59.319401 15197 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:59.320998 15179 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 18:59:59.321166 15179 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/minicluster-data/ts-1/wal
uuid: "3cf97c97242e411a90d80ef88e985a9a"
format_stamp: "Formatted at 2026-08-11 18:59:59 on dist-test-slave-0kls"
server_key: "c3da689303318bddbc5fd4bf4a34e860"
server_key_iv: "5e0ff792b9ef7bce4a7ca5fe02661d32"
server_key_version: "encryptionkey@0"
I20260811 18:59:59.321331 15179 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:59:59.333963 15179 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:59:59.337399 15179 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:59:59.337615 15179 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:59:59.339304 15179 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:59:59.339380 15179 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:59.339450 15179 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:59:59.339495 15179 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 18:59:59 dist-test-slave-0kls krb5kdc[14905](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474799, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.2@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
I20260811 18:59:59.351837 15179 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.2:32807
I20260811 18:59:59.351838 15319 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.2:32807 every 8 connection(s)
I20260811 18:59:59.353045 15179 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
I20260811 18:59:59.354441 15005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:59.340410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.2:49957 (local address 127.13.109.62:43061)
0811 18:59:59.340570 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:59.340575 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:59.341758 (+ 1183us) server_negotiation.cc:407] Connection header received
0811 18:59:59.343419 (+ 1661us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:59.343423 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:59.343490 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:59.343601 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:59.345066 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:59.345613 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:59.346965 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:59.347276 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:59.347398 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:59.350378 (+ 2980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:59.350384 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:59.350389 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:59:59.350432 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:59.352189 (+ 1757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:59.352703 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:59.352707 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:59.352709 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:59.352807 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:59.353147 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:59.353150 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:59.353152 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:59.353415 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:59.353531 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:59.354141 (+ 610us) server_negotiation.cc:299] Negotiation successful
0811 18:59:59.354270 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 18:59:59.355227 15206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:59.340958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:43061 (local address 127.13.109.2:49957)
0811 18:59:59.341563 (+ 605us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:59.341628 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0811 18:59:59.343112 (+ 1484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:59:59.343794 (+ 682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:59:59.343806 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:59:59.344216 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:59:59.344879 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:59.344904 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:59.345752 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:59.345755 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:59.346818 (+ 1063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:59.346830 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:59.347457 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:59.347462 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:59.347643 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:59.348364 (+ 721us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:59:59.348381 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:59:59.350166 (+ 1785us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:59:59.352303 (+ 2137us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:59:59.352308 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:59:59.352320 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:59:59.352573 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:59:59.352902 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:59:59.352904 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:59:59.352906 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:59:59.353032 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:59:59.353535 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:59:59.353541 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:59:59.353861 (+ 320us) client_negotiation.cc:769] Sending connection context
0811 18:59:59.354078 (+ 217us) client_negotiation.cc:240] Negotiation successful
0811 18:59:59.354419 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":192,"threads_started":1}
I20260811 18:59:59.356642 15320 heartbeater.cc:347] Connected to a master server at 127.13.109.62:43061
I20260811 18:59:59.356827 15320 heartbeater.cc:464] Registering TS with master...
I20260811 18:59:59.357364 15320 heartbeater.cc:511] Master 127.13.109.62:43061 requested a full tablet report, sending...
I20260811 18:59:59.358472 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 15179
I20260811 18:59:59.358598 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/minicluster-data/ts-1/wal/instance
I20260811 18:59:59.358606 14952 ts_manager.cc:194] Registered new tserver with Master: 3cf97c97242e411a90d80ef88e985a9a (127.13.109.2:32807)
I20260811 18:59:59.358881 13748 external_mini_cluster.cc:1428] Setting key e9f042b9291ba1f79675fe95601ec24a
I20260811 18:59:59.359445 14952 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.109.2@KRBTEST.COM'} at 127.13.109.2:49957
I20260811 18:59:59.367664 15328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:59.358983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:43061 (local address 127.13.109.2:36499)
0811 18:59:59.359529 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:59.359545 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:59:59.359696 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:59:59.360082 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:59:59.360084 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:59:59.360344 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:59:59.360497 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:59.360506 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:59.361282 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:59.361286 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:59.361940 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:59.361949 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:59.362422 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:59.362427 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:59.362525 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:59.363091 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:59:59.363103 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:59:59.363441 (+ 338us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:59:59.365542 (+ 2101us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:59:59.365545 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:59:59.365547 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:59:59.365843 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:59:59.366191 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:59:59.366194 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:59:59.366195 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:59:59.366314 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:59:59.367199 (+ 885us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:59:59.367201 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:59:59.367289 (+ 88us) client_negotiation.cc:769] Sending connection context
0811 18:59:59.367377 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 18:59:59.367502 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":108,"threads_started":1}
I20260811 18:59:59.367859 15005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:59.359087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.2:36499 (local address 127.13.109.62:43061)
0811 18:59:59.359270 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:59.359275 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:59.359613 (+ 338us) server_negotiation.cc:407] Connection header received
0811 18:59:59.359839 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:59.359842 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:59.359886 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:59.359937 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:59.360629 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:59.361137 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:59.362071 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:59.362294 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:59.362401 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:59.363591 (+ 1190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:59.363596 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:59.363602 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:59:59.363634 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:59.365429 (+ 1795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:59.365954 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:59.365961 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:59.365965 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:59.366074 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:59.366717 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:59.366723 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:59.366727 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:59.367060 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:59.367183 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:59.367485 (+ 302us) server_negotiation.cc:299] Negotiation successful
0811 18:59:59.367640 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
WARNING: no policy specified for kudu/127.13.109.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.109.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.109.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.3.keytab.
Entry for principal kudu/127.13.109.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.3.keytab.
I20260811 18:59:59.392629 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.3:0
--local_ip_for_outbound_sockets=127.13.109.3
--webserver_interface=127.13.109.3
--webserver_port=0
--tserver_master_addrs=127.13.109.62:43061
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.3.keytab
--principal=kudu/127.13.109.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:44071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:59:59.520951 15333 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:59:59.521292 15333 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:59:59.521344 15333 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:59:59.525746 15333 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:59:59.525841 15333 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:59:59.525885 15333 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 18:59:59.525954 15333 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.3
I20260811 18:59:59.532433 15333 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:44071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.3.keytab
--principal=kudu/127.13.109.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.109.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.109.62:43061
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15333
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.3
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:59:59.533749 15333 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:59:59.534998 15333 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:59:59.545785 15341 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:59.545854 15338 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:59.545837 15339 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:59:59.546032 15333 server_base.cc:1034] running on GCE node
I20260811 18:59:59.546584 15333 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:59:59.547279 15333 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:59:59.548473 15333 hybrid_clock.cc:648] HybridClock initialized: now 1786474799548443 us; error 50 us; skew 500 ppm
Aug 11 18:59:59 dist-test-slave-0kls krb5kdc[14905](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474799, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:59:59.551568 15333 init.cc:377] Logged in from keytab as kudu/127.13.109.3@KRBTEST.COM (short username kudu)
I20260811 18:59:59.552501 15333 webserver.cc:460] Webserver started at http://127.13.109.3:33277/ using document root <none> and password file <none>
I20260811 18:59:59.553107 15333 fs_manager.cc:359] Metadata directory not provided
I20260811 18:59:59.553203 15333 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:59:59.553447 15333 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:59:59.555723 15333 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/minicluster-data/ts-2/data/instance:
uuid: "09c6b0f091c6462f9144d47f2ba2b118"
format_stamp: "Formatted at 2026-08-11 18:59:59 on dist-test-slave-0kls"
server_key: "7eb764d9d19306a0eb1a247d6c19a9b1"
server_key_iv: "36767b634c4db2c94bd9cbd369dd6f48"
server_key_version: "encryptionkey@0"
I20260811 18:59:59.556411 15333 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/minicluster-data/ts-2/wal/instance:
uuid: "09c6b0f091c6462f9144d47f2ba2b118"
format_stamp: "Formatted at 2026-08-11 18:59:59 on dist-test-slave-0kls"
server_key: "7eb764d9d19306a0eb1a247d6c19a9b1"
server_key_iv: "36767b634c4db2c94bd9cbd369dd6f48"
server_key_version: "encryptionkey@0"
I20260811 18:59:59.561007 15333 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260811 18:59:59.564435 15348 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:59.566074 15333 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 18:59:59.566234 15333 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/minicluster-data/ts-2/wal
uuid: "09c6b0f091c6462f9144d47f2ba2b118"
format_stamp: "Formatted at 2026-08-11 18:59:59 on dist-test-slave-0kls"
server_key: "7eb764d9d19306a0eb1a247d6c19a9b1"
server_key_iv: "36767b634c4db2c94bd9cbd369dd6f48"
server_key_version: "encryptionkey@0"
I20260811 18:59:59.566371 15333 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:59:59.580469 15333 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:59:59.584172 15333 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:59:59.584403 15333 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:59:59.586151 15333 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:59:59.586243 15333 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:59.586319 15333 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:59:59.586374 15333 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 18:59:59 dist-test-slave-0kls krb5kdc[14905](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474799, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.3@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
I20260811 18:59:59.598121 15333 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.3:43043
I20260811 18:59:59.598173 15469 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.3:43043 every 8 connection(s)
I20260811 18:59:59.599550 15333 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
I20260811 18:59:59.601300 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 15333
I20260811 18:59:59.601425 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/minicluster-data/ts-2/wal/instance
I20260811 18:59:59.601714 13748 external_mini_cluster.cc:1428] Setting key 549d4ef3fbb92c8ac1300e574633839b
I20260811 18:59:59.603834 15005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:59.587064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.3:54833 (local address 127.13.109.62:43061)
0811 18:59:59.587258 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:59.587264 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:59.588613 (+ 1349us) server_negotiation.cc:407] Connection header received
0811 18:59:59.589952 (+ 1339us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:59.589956 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:59.590023 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:59.590110 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:59.591591 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:59.592307 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:59.593692 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:59.594108 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:59.594252 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:59.599125 (+ 4873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:59.599129 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:59.599131 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:59:59.599165 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:59.601139 (+ 1974us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:59.601885 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:59.601891 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:59.601892 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:59.602002 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:59.602478 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:59.602482 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:59.602484 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:59.602770 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:59.602869 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:59.603523 (+ 654us) server_negotiation.cc:299] Negotiation successful
0811 18:59:59.603668 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 18:59:59.604838 15358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:59.587922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:43061 (local address 127.13.109.3:54833)
0811 18:59:59.588439 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:59.588495 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 18:59:59.589650 (+ 1155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:59:59.590364 (+ 714us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:59:59.590372 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:59:59.590734 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:59:59.591371 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:59.591386 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:59.592451 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:59.592455 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:59.593548 (+ 1093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:59.593567 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:59.594312 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:59.594316 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:59.594509 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:59.595772 (+ 1263us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:59:59.595799 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:59:59.597648 (+ 1849us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:59:59.601302 (+ 3654us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:59:59.601308 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:59:59.601321 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:59:59.601712 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:59:59.602139 (+ 427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:59:59.602142 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:59:59.602144 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:59:59.602368 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:59:59.602895 (+ 527us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:59:59.602901 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:59:59.603255 (+ 354us) client_negotiation.cc:769] Sending connection context
0811 18:59:59.603459 (+ 204us) client_negotiation.cc:240] Negotiation successful
0811 18:59:59.603847 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":147,"threads_started":1}
I20260811 18:59:59.606386 15470 heartbeater.cc:347] Connected to a master server at 127.13.109.62:43061
I20260811 18:59:59.606573 15470 heartbeater.cc:464] Registering TS with master...
I20260811 18:59:59.607151 15470 heartbeater.cc:511] Master 127.13.109.62:43061 requested a full tablet report, sending...
I20260811 18:59:59.608346 14952 ts_manager.cc:194] Registered new tserver with Master: 09c6b0f091c6462f9144d47f2ba2b118 (127.13.109.3:43043)
I20260811 18:59:59.609215 14952 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.109.3@KRBTEST.COM'} at 127.13.109.3:54833
I20260811 18:59:59.616469 13748 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 18:59:59.618112 15005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:59.608952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.3:60521 (local address 127.13.109.62:43061)
0811 18:59:59.609102 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:59.609107 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:59.609728 (+ 621us) server_negotiation.cc:407] Connection header received
0811 18:59:59.609924 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:59.609927 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:59.609971 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:59.610045 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:59.610675 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:59.611188 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:59.612455 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:59.612840 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:59.612982 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:59.614436 (+ 1454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:59.614440 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:59.614443 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:59:59.614471 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:59.616257 (+ 1786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:59.616770 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:59.616774 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:59.616776 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:59.616865 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:59.617164 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:59.617169 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:59.617170 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:59.617385 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:59.617497 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:59.617824 (+ 327us) server_negotiation.cc:299] Negotiation successful
0811 18:59:59.617950 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 18:59:59.618129 15477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:59.608832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:43061 (local address 127.13.109.3:60521)
0811 18:59:59.609621 (+ 789us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:59.609636 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 18:59:59.609782 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:59:59.610097 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:59:59.610101 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:59:59.610380 (+ 279us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:59:59.610536 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:59.610546 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:59.611340 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:59.611344 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:59.612270 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:59.612288 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:59.613006 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:59.613010 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:59.613125 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:59.613759 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:59:59.613773 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:59:59.614256 (+ 483us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:59:59.616359 (+ 2103us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:59:59.616364 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:59:59.616366 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:59:59.616651 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:59:59.616964 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:59:59.616968 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:59:59.616970 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:59:59.617068 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:59:59.617489 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:59:59.617492 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:59:59.617604 (+ 112us) client_negotiation.cc:769] Sending connection context
0811 18:59:59.617721 (+ 117us) client_negotiation.cc:240] Negotiation successful
0811 18:59:59.617940 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":703,"thread_start_us":114,"threads_started":1}
Aug 11 18:59:59 dist-test-slave-0kls krb5kdc[14905](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474799, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 18:59:59 dist-test-slave-0kls krb5kdc[14905](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474799, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
I20260811 18:59:59.650743 15005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:59.640449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41614 (local address 127.13.109.62:43061)
0811 18:59:59.640607 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:59.640611 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:59.640627 (+ 16us) server_negotiation.cc:407] Connection header received
0811 18:59:59.640685 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:59.640687 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:59.640735 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:59.640814 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:59.641962 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:59.642604 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:59.643478 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:59.643703 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:59.643786 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:59.646633 (+ 2847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:59.646637 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:59.646639 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:59:59.646677 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:59.648610 (+ 1933us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:59.649181 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:59.649184 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:59.649186 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:59.649280 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:59.649591 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:59.649596 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:59.649597 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:59.649917 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:59.650056 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:59.650367 (+ 311us) server_negotiation.cc:299] Negotiation successful
0811 18:59:59.650488 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 18:59:59.654237 14952 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41614:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:59:59.657649 14952 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:59:59.674525 15490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:59.668011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.1:38161 (local address 127.0.0.1:47876)
0811 18:59:59.668934 (+ 923us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:59.668993 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 18:59:59.669533 (+ 540us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:59:59.670152 (+ 619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:59:59.670156 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:59:59.670197 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:59:59.670377 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:59.670386 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:59.671714 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:59.671717 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:59.672864 (+ 1147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:59.672875 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:59.673856 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:59.673860 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:59.673995 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:59.674063 (+ 68us) client_negotiation.cc:769] Sending connection context
0811 18:59:59.674189 (+ 126us) client_negotiation.cc:240] Negotiation successful
0811 18:59:59.674331 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":726,"mutex_wait_us":149,"thread_start_us":118,"threads_started":1}
I20260811 18:59:59.674657 15489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:59.668230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47876 (local address 127.13.109.1:38161)
0811 18:59:59.669226 (+ 996us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:59.669232 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:59.669255 (+ 23us) server_negotiation.cc:407] Connection header received
0811 18:59:59.669671 (+ 416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:59.669685 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:59.669890 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:59.670010 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:59:59.670516 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:59.671601 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:59.673001 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:59.673696 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:59.673841 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:59.674009 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:59.674392 (+ 383us) server_negotiation.cc:299] Negotiation successful
0811 18:59:59.674515 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":889,"thread_start_us":109,"threads_started":1}
I20260811 18:59:59.675107 15488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:59.668317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.2:32807 (local address 127.0.0.1:41214)
0811 18:59:59.669153 (+ 836us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:59.669166 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:59:59.669300 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:59:59.669911 (+ 611us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:59:59.669918 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:59:59.669942 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:59:59.670118 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:59.670126 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:59.671791 (+ 1665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:59.671795 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:59.673349 (+ 1554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:59.673361 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:59.674675 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:59.674678 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:59.674791 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:59.674820 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 18:59:59.674899 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 18:59:59.674974 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":771,"mutex_wait_us":224,"thread_start_us":97,"threads_started":1}
I20260811 18:59:59.675856 15492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:59.668789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.3:43043 (local address 127.0.0.1:50838)
0811 18:59:59.669563 (+ 774us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:59.669578 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 18:59:59.669656 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:59:59.670229 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:59:59.670232 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:59:59.670244 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:59:59.670398 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:59.670406 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:59.671813 (+ 1407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:59.671816 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:59.673349 (+ 1533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:59.673361 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:59.675462 (+ 2101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:59.675466 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:59.675551 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:59.675570 (+ 19us) client_negotiation.cc:769] Sending connection context
0811 18:59:59.675664 (+ 94us) client_negotiation.cc:240] Negotiation successful
0811 18:59:59.675735 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":707,"mutex_wait_us":113,"thread_start_us":118,"threads_started":1}
I20260811 18:59:59.675935 15491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:59.668761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41214 (local address 127.13.109.2:32807)
0811 18:59:59.669213 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:59.669227 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:59.669255 (+ 28us) server_negotiation.cc:407] Connection header received
0811 18:59:59.669493 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:59.669499 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:59.669728 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:59.669980 (+ 252us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:59:59.670310 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:59.671652 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:59.673517 (+ 1865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:59.674342 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:59.674446 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:59.674612 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:59.674935 (+ 323us) server_negotiation.cc:299] Negotiation successful
0811 18:59:59.675799 (+ 864us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":120,"threads_started":1}
I20260811 18:59:59.676090 15493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:59.669171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50838 (local address 127.13.109.3:43043)
0811 18:59:59.669520 (+ 349us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:59.669531 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:59.669737 (+ 206us) server_negotiation.cc:407] Connection header received
0811 18:59:59.669882 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:59.669888 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:59.670089 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:59.670248 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:59:59.670531 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:59.671704 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:59.673489 (+ 1785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:59.675110 (+ 1621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:59.675226 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:59.675392 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:59.675708 (+ 316us) server_negotiation.cc:299] Negotiation successful
0811 18:59:59.675804 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":245,"thread_start_us":123,"threads_started":1}
I20260811 18:59:59.678257 15097 tablet_service.cc:1467] Processing CreateTablet for tablet 176a5c272b7148d1803feba06d6780ac (DEFAULT_TABLE table=test-table [id=9fee03fce20c43438988407b12e2ecc3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:59:59.678377 15251 tablet_service.cc:1467] Processing CreateTablet for tablet 176a5c272b7148d1803feba06d6780ac (DEFAULT_TABLE table=test-table [id=9fee03fce20c43438988407b12e2ecc3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:59:59.678689 15398 tablet_service.cc:1467] Processing CreateTablet for tablet 176a5c272b7148d1803feba06d6780ac (DEFAULT_TABLE table=test-table [id=9fee03fce20c43438988407b12e2ecc3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:59:59.679574 15097 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 176a5c272b7148d1803feba06d6780ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:59:59.679574 15251 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 176a5c272b7148d1803feba06d6780ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:59:59.679615 15398 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 176a5c272b7148d1803feba06d6780ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:59:59.687105 15494 tablet_bootstrap.cc:492] T 176a5c272b7148d1803feba06d6780ac P 3cf97c97242e411a90d80ef88e985a9a: Bootstrap starting.
I20260811 18:59:59.687983 15495 tablet_bootstrap.cc:492] T 176a5c272b7148d1803feba06d6780ac P bbfd45351bfd41879acc3a7d6073fc55: Bootstrap starting.
I20260811 18:59:59.688802 15497 tablet_bootstrap.cc:492] T 176a5c272b7148d1803feba06d6780ac P 09c6b0f091c6462f9144d47f2ba2b118: Bootstrap starting.
I20260811 18:59:59.689819 15494 tablet_bootstrap.cc:654] T 176a5c272b7148d1803feba06d6780ac P 3cf97c97242e411a90d80ef88e985a9a: Neither blocks nor log segments found. Creating new log.
I20260811 18:59:59.690164 15495 tablet_bootstrap.cc:654] T 176a5c272b7148d1803feba06d6780ac P bbfd45351bfd41879acc3a7d6073fc55: Neither blocks nor log segments found. Creating new log.
I20260811 18:59:59.690743 15494 log.cc:824] T 176a5c272b7148d1803feba06d6780ac P 3cf97c97242e411a90d80ef88e985a9a: Log is configured to *not* fsync() on all Append() calls
I20260811 18:59:59.690922 15497 tablet_bootstrap.cc:654] T 176a5c272b7148d1803feba06d6780ac P 09c6b0f091c6462f9144d47f2ba2b118: Neither blocks nor log segments found. Creating new log.
I20260811 18:59:59.690974 15495 log.cc:824] T 176a5c272b7148d1803feba06d6780ac P bbfd45351bfd41879acc3a7d6073fc55: Log is configured to *not* fsync() on all Append() calls
I20260811 18:59:59.691788 15497 log.cc:824] T 176a5c272b7148d1803feba06d6780ac P 09c6b0f091c6462f9144d47f2ba2b118: Log is configured to *not* fsync() on all Append() calls
I20260811 18:59:59.693048 15494 tablet_bootstrap.cc:492] T 176a5c272b7148d1803feba06d6780ac P 3cf97c97242e411a90d80ef88e985a9a: No bootstrap required, opened a new log
I20260811 18:59:59.693048 15495 tablet_bootstrap.cc:492] T 176a5c272b7148d1803feba06d6780ac P bbfd45351bfd41879acc3a7d6073fc55: No bootstrap required, opened a new log
I20260811 18:59:59.693265 15494 ts_tablet_manager.cc:1397] T 176a5c272b7148d1803feba06d6780ac P 3cf97c97242e411a90d80ef88e985a9a: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 18:59:59.693261 15495 ts_tablet_manager.cc:1397] T 176a5c272b7148d1803feba06d6780ac P bbfd45351bfd41879acc3a7d6073fc55: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 18:59:59.693594 15497 tablet_bootstrap.cc:492] T 176a5c272b7148d1803feba06d6780ac P 09c6b0f091c6462f9144d47f2ba2b118: No bootstrap required, opened a new log
I20260811 18:59:59.693871 15497 ts_tablet_manager.cc:1397] T 176a5c272b7148d1803feba06d6780ac P 09c6b0f091c6462f9144d47f2ba2b118: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260811 18:59:59.697634 15497 raft_consensus.cc:348] T 176a5c272b7148d1803feba06d6780ac P 09c6b0f091c6462f9144d47f2ba2b118 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbfd45351bfd41879acc3a7d6073fc55" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 38161 } } peers { permanent_uuid: "09c6b0f091c6462f9144d47f2ba2b118" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43043 } } peers { permanent_uuid: "3cf97c97242e411a90d80ef88e985a9a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 32807 } }
I20260811 18:59:59.697660 15494 raft_consensus.cc:348] T 176a5c272b7148d1803feba06d6780ac P 3cf97c97242e411a90d80ef88e985a9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbfd45351bfd41879acc3a7d6073fc55" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 38161 } } peers { permanent_uuid: "09c6b0f091c6462f9144d47f2ba2b118" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43043 } } peers { permanent_uuid: "3cf97c97242e411a90d80ef88e985a9a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 32807 } }
I20260811 18:59:59.698004 15494 raft_consensus.cc:374] T 176a5c272b7148d1803feba06d6780ac P 3cf97c97242e411a90d80ef88e985a9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:59:59.698005 15497 raft_consensus.cc:374] T 176a5c272b7148d1803feba06d6780ac P 09c6b0f091c6462f9144d47f2ba2b118 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:59:59.698071 15497 raft_consensus.cc:729] T 176a5c272b7148d1803feba06d6780ac P 09c6b0f091c6462f9144d47f2ba2b118 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09c6b0f091c6462f9144d47f2ba2b118, State: Initialized, Role: FOLLOWER
I20260811 18:59:59.698071 15494 raft_consensus.cc:729] T 176a5c272b7148d1803feba06d6780ac P 3cf97c97242e411a90d80ef88e985a9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3cf97c97242e411a90d80ef88e985a9a, State: Initialized, Role: FOLLOWER
I20260811 18:59:59.698643 15497 consensus_queue.cc:260] T 176a5c272b7148d1803feba06d6780ac P 09c6b0f091c6462f9144d47f2ba2b118 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbfd45351bfd41879acc3a7d6073fc55" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 38161 } } peers { permanent_uuid: "09c6b0f091c6462f9144d47f2ba2b118" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43043 } } peers { permanent_uuid: "3cf97c97242e411a90d80ef88e985a9a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 32807 } }
I20260811 18:59:59.698643 15494 consensus_queue.cc:260] T 176a5c272b7148d1803feba06d6780ac P 3cf97c97242e411a90d80ef88e985a9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbfd45351bfd41879acc3a7d6073fc55" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 38161 } } peers { permanent_uuid: "09c6b0f091c6462f9144d47f2ba2b118" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43043 } } peers { permanent_uuid: "3cf97c97242e411a90d80ef88e985a9a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 32807 } }
I20260811 18:59:59.699667 15320 heartbeater.cc:503] Master 127.13.109.62:43061 was elected leader, sending a full tablet report...
I20260811 18:59:59.699667 15470 heartbeater.cc:503] Master 127.13.109.62:43061 was elected leader, sending a full tablet report...
I20260811 18:59:59.699887 15494 ts_tablet_manager.cc:1428] T 176a5c272b7148d1803feba06d6780ac P 3cf97c97242e411a90d80ef88e985a9a: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260811 18:59:59.699776 15495 raft_consensus.cc:348] T 176a5c272b7148d1803feba06d6780ac P bbfd45351bfd41879acc3a7d6073fc55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbfd45351bfd41879acc3a7d6073fc55" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 38161 } } peers { permanent_uuid: "09c6b0f091c6462f9144d47f2ba2b118" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43043 } } peers { permanent_uuid: "3cf97c97242e411a90d80ef88e985a9a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 32807 } }
I20260811 18:59:59.700078 15495 raft_consensus.cc:374] T 176a5c272b7148d1803feba06d6780ac P bbfd45351bfd41879acc3a7d6073fc55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:59:59.700155 15495 raft_consensus.cc:729] T 176a5c272b7148d1803feba06d6780ac P bbfd45351bfd41879acc3a7d6073fc55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbfd45351bfd41879acc3a7d6073fc55, State: Initialized, Role: FOLLOWER
I20260811 18:59:59.700560 15495 consensus_queue.cc:260] T 176a5c272b7148d1803feba06d6780ac P bbfd45351bfd41879acc3a7d6073fc55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbfd45351bfd41879acc3a7d6073fc55" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 38161 } } peers { permanent_uuid: "09c6b0f091c6462f9144d47f2ba2b118" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43043 } } peers { permanent_uuid: "3cf97c97242e411a90d80ef88e985a9a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 32807 } }
I20260811 18:59:59.701014 15497 ts_tablet_manager.cc:1428] T 176a5c272b7148d1803feba06d6780ac P 09c6b0f091c6462f9144d47f2ba2b118: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260811 18:59:59.701514 15495 ts_tablet_manager.cc:1428] T 176a5c272b7148d1803feba06d6780ac P bbfd45351bfd41879acc3a7d6073fc55: Time spent starting tablet: real 0.008s user 0.008s sys 0.000s
I20260811 18:59:59.701807 15168 heartbeater.cc:503] Master 127.13.109.62:43061 was elected leader, sending a full tablet report...
I20260811 18:59:59.718894 15503 raft_consensus.cc:482] T 176a5c272b7148d1803feba06d6780ac P bbfd45351bfd41879acc3a7d6073fc55 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:59:59.719137 15503 raft_consensus.cc:504] T 176a5c272b7148d1803feba06d6780ac P bbfd45351bfd41879acc3a7d6073fc55 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbfd45351bfd41879acc3a7d6073fc55" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 38161 } } peers { permanent_uuid: "09c6b0f091c6462f9144d47f2ba2b118" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43043 } } peers { permanent_uuid: "3cf97c97242e411a90d80ef88e985a9a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 32807 } }
I20260811 18:59:59.720471 15503 leader_election.cc:288] T 176a5c272b7148d1803feba06d6780ac P bbfd45351bfd41879acc3a7d6073fc55 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 09c6b0f091c6462f9144d47f2ba2b118 (127.13.109.3:43043), 3cf97c97242e411a90d80ef88e985a9a (127.13.109.2:32807)
I20260811 18:59:59.724802 15493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:59.720930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.1:59887 (local address 127.13.109.3:43043)
0811 18:59:59.721061 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:59.721067 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:59.721465 (+ 398us) server_negotiation.cc:407] Connection header received
0811 18:59:59.721620 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:59.721626 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:59.721673 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:59.721761 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:59:59.722093 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:59.722616 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:59.723749 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:59.724149 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:59.724283 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:59.724343 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:59.724564 (+ 221us) server_negotiation.cc:299] Negotiation successful
0811 18:59:59.724647 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 18:59:59.724808 15491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:59.720888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.1:47725 (local address 127.13.109.2:32807)
0811 18:59:59.721033 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:59.721039 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:59.721359 (+ 320us) server_negotiation.cc:407] Connection header received
0811 18:59:59.721620 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:59.721623 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:59.721671 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:59.721761 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:59:59.722153 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:59.722660 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:59.723768 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:59.724168 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:59.724303 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:59.724360 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:59.724566 (+ 206us) server_negotiation.cc:299] Negotiation successful
0811 18:59:59.724670 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 18:59:59.724865 15505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:59.720786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.3:43043 (local address 127.13.109.1:59887)
0811 18:59:59.721348 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:59.721365 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 18:59:59.721507 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:59:59.721808 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:59:59.721811 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:59:59.721828 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:59:59.721981 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:59.721987 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:59.722736 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:59.722741 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:59.723617 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:59.723627 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:59.724278 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:59.724281 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:59.724378 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:59.724456 (+ 78us) client_negotiation.cc:769] Sending connection context
0811 18:59:59.724546 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 18:59:59.724640 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"mutex_wait_us":57,"spinlock_wait_cycles":16256,"thread_start_us":134,"threads_started":1}
I20260811 18:59:59.725128 15504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:59.720768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.2:32807 (local address 127.13.109.1:47725)
0811 18:59:59.721236 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:59.721249 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:59:59.721370 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:59:59.721818 (+ 448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:59:59.721821 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:59:59.721840 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:59:59.722000 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:59.722006 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:59.722777 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:59.722782 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:59.723636 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:59.723646 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:59.724295 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:59.724299 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:59.724428 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:59.724459 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 18:59:59.724547 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 18:59:59.724642 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":404,"mutex_wait_us":77,"thread_start_us":161,"threads_started":1}
I20260811 18:59:59.725852 15421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "176a5c272b7148d1803feba06d6780ac" candidate_uuid: "bbfd45351bfd41879acc3a7d6073fc55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09c6b0f091c6462f9144d47f2ba2b118" is_pre_election: true
I20260811 18:59:59.725878 15253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "176a5c272b7148d1803feba06d6780ac" candidate_uuid: "bbfd45351bfd41879acc3a7d6073fc55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3cf97c97242e411a90d80ef88e985a9a" is_pre_election: true
I20260811 18:59:59.726207 15421 raft_consensus.cc:2454] T 176a5c272b7148d1803feba06d6780ac P 09c6b0f091c6462f9144d47f2ba2b118 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bbfd45351bfd41879acc3a7d6073fc55 in term 0.
I20260811 18:59:59.726207 15253 raft_consensus.cc:2454] T 176a5c272b7148d1803feba06d6780ac P 3cf97c97242e411a90d80ef88e985a9a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bbfd45351bfd41879acc3a7d6073fc55 in term 0.
I20260811 18:59:59.726739 15048 leader_election.cc:302] T 176a5c272b7148d1803feba06d6780ac P bbfd45351bfd41879acc3a7d6073fc55 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 09c6b0f091c6462f9144d47f2ba2b118, 3cf97c97242e411a90d80ef88e985a9a, bbfd45351bfd41879acc3a7d6073fc55; no voters:
I20260811 18:59:59.727048 15503 raft_consensus.cc:2781] T 176a5c272b7148d1803feba06d6780ac P bbfd45351bfd41879acc3a7d6073fc55 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:59:59.727187 15503 raft_consensus.cc:482] T 176a5c272b7148d1803feba06d6780ac P bbfd45351bfd41879acc3a7d6073fc55 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:59:59.727249 15503 raft_consensus.cc:3037] T 176a5c272b7148d1803feba06d6780ac P bbfd45351bfd41879acc3a7d6073fc55 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:59:59.728363 15503 raft_consensus.cc:504] T 176a5c272b7148d1803feba06d6780ac P bbfd45351bfd41879acc3a7d6073fc55 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbfd45351bfd41879acc3a7d6073fc55" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 38161 } } peers { permanent_uuid: "09c6b0f091c6462f9144d47f2ba2b118" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43043 } } peers { permanent_uuid: "3cf97c97242e411a90d80ef88e985a9a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 32807 } }
I20260811 18:59:59.728854 15503 leader_election.cc:288] T 176a5c272b7148d1803feba06d6780ac P bbfd45351bfd41879acc3a7d6073fc55 [CANDIDATE]: Term 1 election: Requested vote from peers 09c6b0f091c6462f9144d47f2ba2b118 (127.13.109.3:43043), 3cf97c97242e411a90d80ef88e985a9a (127.13.109.2:32807)
I20260811 18:59:59.729228 15253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "176a5c272b7148d1803feba06d6780ac" candidate_uuid: "bbfd45351bfd41879acc3a7d6073fc55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3cf97c97242e411a90d80ef88e985a9a"
I20260811 18:59:59.729286 15421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "176a5c272b7148d1803feba06d6780ac" candidate_uuid: "bbfd45351bfd41879acc3a7d6073fc55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09c6b0f091c6462f9144d47f2ba2b118"
I20260811 18:59:59.729368 15253 raft_consensus.cc:3037] T 176a5c272b7148d1803feba06d6780ac P 3cf97c97242e411a90d80ef88e985a9a [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:59:59.729399 15421 raft_consensus.cc:3037] T 176a5c272b7148d1803feba06d6780ac P 09c6b0f091c6462f9144d47f2ba2b118 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:59:59.730836 15421 raft_consensus.cc:2454] T 176a5c272b7148d1803feba06d6780ac P 09c6b0f091c6462f9144d47f2ba2b118 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bbfd45351bfd41879acc3a7d6073fc55 in term 1.
I20260811 18:59:59.731216 15253 raft_consensus.cc:2454] T 176a5c272b7148d1803feba06d6780ac P 3cf97c97242e411a90d80ef88e985a9a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bbfd45351bfd41879acc3a7d6073fc55 in term 1.
I20260811 18:59:59.731237 15050 leader_election.cc:302] T 176a5c272b7148d1803feba06d6780ac P bbfd45351bfd41879acc3a7d6073fc55 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09c6b0f091c6462f9144d47f2ba2b118, bbfd45351bfd41879acc3a7d6073fc55; no voters:
I20260811 18:59:59.731541 15503 raft_consensus.cc:2781] T 176a5c272b7148d1803feba06d6780ac P bbfd45351bfd41879acc3a7d6073fc55 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:59:59.731767 15503 raft_consensus.cc:686] T 176a5c272b7148d1803feba06d6780ac P bbfd45351bfd41879acc3a7d6073fc55 [term 1 LEADER]: Becoming Leader. State: Replica: bbfd45351bfd41879acc3a7d6073fc55, State: Running, Role: LEADER
I20260811 18:59:59.732147 15503 consensus_queue.cc:237] T 176a5c272b7148d1803feba06d6780ac P bbfd45351bfd41879acc3a7d6073fc55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbfd45351bfd41879acc3a7d6073fc55" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 38161 } } peers { permanent_uuid: "09c6b0f091c6462f9144d47f2ba2b118" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43043 } } peers { permanent_uuid: "3cf97c97242e411a90d80ef88e985a9a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 32807 } }
I20260811 18:59:59.735603 14953 catalog_manager.cc:5495] T 176a5c272b7148d1803feba06d6780ac P bbfd45351bfd41879acc3a7d6073fc55 reported cstate change: term changed from 0 to 1, leader changed from <none> to bbfd45351bfd41879acc3a7d6073fc55 (127.13.109.1). New cstate: current_term: 1 leader_uuid: "bbfd45351bfd41879acc3a7d6073fc55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbfd45351bfd41879acc3a7d6073fc55" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 38161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09c6b0f091c6462f9144d47f2ba2b118" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43043 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3cf97c97242e411a90d80ef88e985a9a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 32807 } health_report { overall_health: UNKNOWN } } }
I20260811 18:59:59.761436 15489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:59.757402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47880 (local address 127.13.109.1:38161)
0811 18:59:59.757564 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:59.757568 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:59.757582 (+ 14us) server_negotiation.cc:407] Connection header received
0811 18:59:59.757819 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:59.757822 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:59.757884 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:59.757974 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:59:59.758270 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:59.758778 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:59.759714 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:59.759935 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:59.760014 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:59.760515 (+ 501us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:59:59.760995 (+ 480us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:59:59.761088 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:59.761242 (+ 154us) server_negotiation.cc:299] Negotiation successful
0811 18:59:59.761306 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 18:59:59.768324 15421 raft_consensus.cc:1261] T 176a5c272b7148d1803feba06d6780ac P 09c6b0f091c6462f9144d47f2ba2b118 [term 1 FOLLOWER]: Refusing update from remote peer bbfd45351bfd41879acc3a7d6073fc55: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:59:59.768324 15253 raft_consensus.cc:1261] T 176a5c272b7148d1803feba06d6780ac P 3cf97c97242e411a90d80ef88e985a9a [term 1 FOLLOWER]: Refusing update from remote peer bbfd45351bfd41879acc3a7d6073fc55: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:59:59.769155 15503 consensus_queue.cc:1035] T 176a5c272b7148d1803feba06d6780ac P bbfd45351bfd41879acc3a7d6073fc55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09c6b0f091c6462f9144d47f2ba2b118" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:59:59.769369 15506 consensus_queue.cc:1035] T 176a5c272b7148d1803feba06d6780ac P bbfd45351bfd41879acc3a7d6073fc55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3cf97c97242e411a90d80ef88e985a9a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 32807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:59:59.779193 15511 mvcc.cc:205] Tried to move back new op lower bound from 7317400779842211840 to 7317400779705810944. Current Snapshot: MvccSnapshot[applied={T|T < 7317400779842211840}]
I20260811 18:59:59.792555 14953 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41614:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:59:59.792846 14953 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41614:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:59:59.795928 14953 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1f76e5475fab4ce384e2d95956b9aa32: Sending DeleteTablet for 3 replicas of tablet 176a5c272b7148d1803feba06d6780ac
I20260811 18:59:59.797116 15251 tablet_service.cc:1514] Processing DeleteTablet for tablet 176a5c272b7148d1803feba06d6780ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:59:59 UTC) from {username='kudu', principal='kudu/127.13.109.62@KRBTEST.COM'} at 127.0.0.1:41214
I20260811 18:59:59.797277 15097 tablet_service.cc:1514] Processing DeleteTablet for tablet 176a5c272b7148d1803feba06d6780ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:59:59 UTC) from {username='kudu', principal='kudu/127.13.109.62@KRBTEST.COM'} at 127.0.0.1:47876
I20260811 18:59:59.797928 15520 tablet_replica.cc:330] T 176a5c272b7148d1803feba06d6780ac P 3cf97c97242e411a90d80ef88e985a9a: stopping tablet replica
I20260811 18:59:59.797873 15398 tablet_service.cc:1514] Processing DeleteTablet for tablet 176a5c272b7148d1803feba06d6780ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:59:59 UTC) from {username='kudu', principal='kudu/127.13.109.62@KRBTEST.COM'} at 127.0.0.1:50838
I20260811 18:59:59.798218 14953 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41614:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:59:59.798414 15521 tablet_replica.cc:330] T 176a5c272b7148d1803feba06d6780ac P bbfd45351bfd41879acc3a7d6073fc55: stopping tablet replica
I20260811 18:59:59.798612 15520 raft_consensus.cc:2229] T 176a5c272b7148d1803feba06d6780ac P 3cf97c97242e411a90d80ef88e985a9a [term 1 FOLLOWER]: Raft consensus shutting down.
W20260811 18:59:59.798714 14953 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:59:59.798729 15521 raft_consensus.cc:2229] T 176a5c272b7148d1803feba06d6780ac P bbfd45351bfd41879acc3a7d6073fc55 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:59:59.799170 15520 raft_consensus.cc:2258] T 176a5c272b7148d1803feba06d6780ac P 3cf97c97242e411a90d80ef88e985a9a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:59:59.799223 15521 raft_consensus.cc:2258] T 176a5c272b7148d1803feba06d6780ac P bbfd45351bfd41879acc3a7d6073fc55 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:59:59.799760 15522 tablet_replica.cc:330] T 176a5c272b7148d1803feba06d6780ac P 09c6b0f091c6462f9144d47f2ba2b118: stopping tablet replica
I20260811 18:59:59.800127 15522 raft_consensus.cc:2229] T 176a5c272b7148d1803feba06d6780ac P 09c6b0f091c6462f9144d47f2ba2b118 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:59:59.800529 15522 raft_consensus.cc:2258] T 176a5c272b7148d1803feba06d6780ac P 09c6b0f091c6462f9144d47f2ba2b118 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:59:59.801116 15520 ts_tablet_manager.cc:1905] T 176a5c272b7148d1803feba06d6780ac P 3cf97c97242e411a90d80ef88e985a9a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:59:59.802199 15522 ts_tablet_manager.cc:1905] T 176a5c272b7148d1803feba06d6780ac P 09c6b0f091c6462f9144d47f2ba2b118: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:59:59.804548 15097 tablet_service.cc:1467] Processing CreateTablet for tablet d886abbc036e4c438a5ae122d1ad641f (DEFAULT_TABLE table=test-table [id=d4aae8c4958142d69688388dfa4523d5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:59:59.804733 15398 tablet_service.cc:1467] Processing CreateTablet for tablet d886abbc036e4c438a5ae122d1ad641f (DEFAULT_TABLE table=test-table [id=d4aae8c4958142d69688388dfa4523d5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:59:59.804926 15097 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d886abbc036e4c438a5ae122d1ad641f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:59:59.805023 15398 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d886abbc036e4c438a5ae122d1ad641f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:59:59.805366 15520 ts_tablet_manager.cc:1918] T 176a5c272b7148d1803feba06d6780ac P 3cf97c97242e411a90d80ef88e985a9a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:59:59.805527 15520 log.cc:1196] T 176a5c272b7148d1803feba06d6780ac P 3cf97c97242e411a90d80ef88e985a9a: Deleting WAL directory at /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/minicluster-data/ts-1/wal/wals/176a5c272b7148d1803feba06d6780ac
I20260811 18:59:59.805960 15520 ts_tablet_manager.cc:1939] T 176a5c272b7148d1803feba06d6780ac P 3cf97c97242e411a90d80ef88e985a9a: Deleting consensus metadata
I20260811 18:59:59.806070 15251 tablet_service.cc:1467] Processing CreateTablet for tablet d886abbc036e4c438a5ae122d1ad641f (DEFAULT_TABLE table=test-table [id=d4aae8c4958142d69688388dfa4523d5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:59:59.806357 15251 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d886abbc036e4c438a5ae122d1ad641f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:59:59.807685 14937 catalog_manager.cc:4841] TS 3cf97c97242e411a90d80ef88e985a9a (127.13.109.2:32807): tablet 176a5c272b7148d1803feba06d6780ac (table test-table [id=9fee03fce20c43438988407b12e2ecc3]) successfully deleted
I20260811 18:59:59.807746 15495 tablet_bootstrap.cc:492] T d886abbc036e4c438a5ae122d1ad641f P bbfd45351bfd41879acc3a7d6073fc55: Bootstrap starting.
I20260811 18:59:59.808655 15494 tablet_bootstrap.cc:492] T d886abbc036e4c438a5ae122d1ad641f P 3cf97c97242e411a90d80ef88e985a9a: Bootstrap starting.
I20260811 18:59:59.808835 15495 tablet_bootstrap.cc:654] T d886abbc036e4c438a5ae122d1ad641f P bbfd45351bfd41879acc3a7d6073fc55: Neither blocks nor log segments found. Creating new log.
I20260811 18:59:59.809765 15494 tablet_bootstrap.cc:654] T d886abbc036e4c438a5ae122d1ad641f P 3cf97c97242e411a90d80ef88e985a9a: Neither blocks nor log segments found. Creating new log.
I20260811 18:59:59.810207 15495 tablet_bootstrap.cc:492] T d886abbc036e4c438a5ae122d1ad641f P bbfd45351bfd41879acc3a7d6073fc55: No bootstrap required, opened a new log
I20260811 18:59:59.810308 15495 ts_tablet_manager.cc:1397] T d886abbc036e4c438a5ae122d1ad641f P bbfd45351bfd41879acc3a7d6073fc55: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 18:59:59.810474 15497 tablet_bootstrap.cc:492] T d886abbc036e4c438a5ae122d1ad641f P 09c6b0f091c6462f9144d47f2ba2b118: Bootstrap starting.
I20260811 18:59:59.810837 15495 raft_consensus.cc:348] T d886abbc036e4c438a5ae122d1ad641f P bbfd45351bfd41879acc3a7d6073fc55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbfd45351bfd41879acc3a7d6073fc55" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 38161 } } peers { permanent_uuid: "3cf97c97242e411a90d80ef88e985a9a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 32807 } } peers { permanent_uuid: "09c6b0f091c6462f9144d47f2ba2b118" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43043 } }
I20260811 18:59:59.810966 15495 raft_consensus.cc:374] T d886abbc036e4c438a5ae122d1ad641f P bbfd45351bfd41879acc3a7d6073fc55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:59:59.811010 15495 raft_consensus.cc:729] T d886abbc036e4c438a5ae122d1ad641f P bbfd45351bfd41879acc3a7d6073fc55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbfd45351bfd41879acc3a7d6073fc55, State: Initialized, Role: FOLLOWER
I20260811 18:59:59.805047 15522 ts_tablet_manager.cc:1918] T 176a5c272b7148d1803feba06d6780ac P 09c6b0f091c6462f9144d47f2ba2b118: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:59:59.811154 15495 consensus_queue.cc:260] T d886abbc036e4c438a5ae122d1ad641f P bbfd45351bfd41879acc3a7d6073fc55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbfd45351bfd41879acc3a7d6073fc55" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 38161 } } peers { permanent_uuid: "3cf97c97242e411a90d80ef88e985a9a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 32807 } } peers { permanent_uuid: "09c6b0f091c6462f9144d47f2ba2b118" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43043 } }
I20260811 18:59:59.811213 15522 log.cc:1196] T 176a5c272b7148d1803feba06d6780ac P 09c6b0f091c6462f9144d47f2ba2b118: Deleting WAL directory at /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/minicluster-data/ts-2/wal/wals/176a5c272b7148d1803feba06d6780ac
I20260811 18:59:59.811430 15495 ts_tablet_manager.cc:1428] T d886abbc036e4c438a5ae122d1ad641f P bbfd45351bfd41879acc3a7d6073fc55: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260811 18:59:59.811553 15497 tablet_bootstrap.cc:654] T d886abbc036e4c438a5ae122d1ad641f P 09c6b0f091c6462f9144d47f2ba2b118: Neither blocks nor log segments found. Creating new log.
I20260811 18:59:59.811774 15522 ts_tablet_manager.cc:1939] T 176a5c272b7148d1803feba06d6780ac P 09c6b0f091c6462f9144d47f2ba2b118: Deleting consensus metadata
I20260811 18:59:59.813264 14939 catalog_manager.cc:4841] TS 09c6b0f091c6462f9144d47f2ba2b118 (127.13.109.3:43043): tablet 176a5c272b7148d1803feba06d6780ac (table test-table [id=9fee03fce20c43438988407b12e2ecc3]) successfully deleted
I20260811 18:59:59.813931 15497 tablet_bootstrap.cc:492] T d886abbc036e4c438a5ae122d1ad641f P 09c6b0f091c6462f9144d47f2ba2b118: No bootstrap required, opened a new log
I20260811 18:59:59.814026 15497 ts_tablet_manager.cc:1397] T d886abbc036e4c438a5ae122d1ad641f P 09c6b0f091c6462f9144d47f2ba2b118: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260811 18:59:59.814030 15521 ts_tablet_manager.cc:1905] T 176a5c272b7148d1803feba06d6780ac P bbfd45351bfd41879acc3a7d6073fc55: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:59:59.814489 15497 raft_consensus.cc:348] T d886abbc036e4c438a5ae122d1ad641f P 09c6b0f091c6462f9144d47f2ba2b118 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbfd45351bfd41879acc3a7d6073fc55" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 38161 } } peers { permanent_uuid: "3cf97c97242e411a90d80ef88e985a9a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 32807 } } peers { permanent_uuid: "09c6b0f091c6462f9144d47f2ba2b118" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43043 } }
I20260811 18:59:59.814626 15497 raft_consensus.cc:374] T d886abbc036e4c438a5ae122d1ad641f P 09c6b0f091c6462f9144d47f2ba2b118 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:59:59.814662 15497 raft_consensus.cc:729] T d886abbc036e4c438a5ae122d1ad641f P 09c6b0f091c6462f9144d47f2ba2b118 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09c6b0f091c6462f9144d47f2ba2b118, State: Initialized, Role: FOLLOWER
I20260811 18:59:59.814798 15497 consensus_queue.cc:260] T d886abbc036e4c438a5ae122d1ad641f P 09c6b0f091c6462f9144d47f2ba2b118 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbfd45351bfd41879acc3a7d6073fc55" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 38161 } } peers { permanent_uuid: "3cf97c97242e411a90d80ef88e985a9a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 32807 } } peers { permanent_uuid: "09c6b0f091c6462f9144d47f2ba2b118" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43043 } }
I20260811 18:59:59.815047 15497 ts_tablet_manager.cc:1428] T d886abbc036e4c438a5ae122d1ad641f P 09c6b0f091c6462f9144d47f2ba2b118: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 18:59:59.816815 15521 ts_tablet_manager.cc:1918] T 176a5c272b7148d1803feba06d6780ac P bbfd45351bfd41879acc3a7d6073fc55: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:59:59.816911 15521 log.cc:1196] T 176a5c272b7148d1803feba06d6780ac P bbfd45351bfd41879acc3a7d6073fc55: Deleting WAL directory at /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/minicluster-data/ts-0/wal/wals/176a5c272b7148d1803feba06d6780ac
I20260811 18:59:59.817265 15521 ts_tablet_manager.cc:1939] T 176a5c272b7148d1803feba06d6780ac P bbfd45351bfd41879acc3a7d6073fc55: Deleting consensus metadata
I20260811 18:59:59.817996 15494 tablet_bootstrap.cc:492] T d886abbc036e4c438a5ae122d1ad641f P 3cf97c97242e411a90d80ef88e985a9a: No bootstrap required, opened a new log
I20260811 18:59:59.818161 15494 ts_tablet_manager.cc:1397] T d886abbc036e4c438a5ae122d1ad641f P 3cf97c97242e411a90d80ef88e985a9a: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.002s
I20260811 18:59:59.818604 15494 raft_consensus.cc:348] T d886abbc036e4c438a5ae122d1ad641f P 3cf97c97242e411a90d80ef88e985a9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbfd45351bfd41879acc3a7d6073fc55" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 38161 } } peers { permanent_uuid: "3cf97c97242e411a90d80ef88e985a9a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 32807 } } peers { permanent_uuid: "09c6b0f091c6462f9144d47f2ba2b118" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43043 } }
I20260811 18:59:59.818722 15494 raft_consensus.cc:374] T d886abbc036e4c438a5ae122d1ad641f P 3cf97c97242e411a90d80ef88e985a9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:59:59.818765 15494 raft_consensus.cc:729] T d886abbc036e4c438a5ae122d1ad641f P 3cf97c97242e411a90d80ef88e985a9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3cf97c97242e411a90d80ef88e985a9a, State: Initialized, Role: FOLLOWER
I20260811 18:59:59.818913 15494 consensus_queue.cc:260] T d886abbc036e4c438a5ae122d1ad641f P 3cf97c97242e411a90d80ef88e985a9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbfd45351bfd41879acc3a7d6073fc55" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 38161 } } peers { permanent_uuid: "3cf97c97242e411a90d80ef88e985a9a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 32807 } } peers { permanent_uuid: "09c6b0f091c6462f9144d47f2ba2b118" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43043 } }
I20260811 18:59:59.819247 15494 ts_tablet_manager.cc:1428] T d886abbc036e4c438a5ae122d1ad641f P 3cf97c97242e411a90d80ef88e985a9a: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260811 18:59:59.821012 14939 catalog_manager.cc:4841] TS bbfd45351bfd41879acc3a7d6073fc55 (127.13.109.1:38161): tablet 176a5c272b7148d1803feba06d6780ac (table test-table [id=9fee03fce20c43438988407b12e2ecc3]) successfully deleted
W20260811 18:59:59.851066 15471 tablet.cc:2369] T d886abbc036e4c438a5ae122d1ad641f P 09c6b0f091c6462f9144d47f2ba2b118: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 18:59:59.854622 15322 tablet.cc:2369] T d886abbc036e4c438a5ae122d1ad641f P 3cf97c97242e411a90d80ef88e985a9a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 18:59:59.872907 15169 tablet.cc:2369] T d886abbc036e4c438a5ae122d1ad641f P bbfd45351bfd41879acc3a7d6073fc55: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:59:59.963896 15501 raft_consensus.cc:482] T d886abbc036e4c438a5ae122d1ad641f P 3cf97c97242e411a90d80ef88e985a9a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:59:59.964095 15501 raft_consensus.cc:504] T d886abbc036e4c438a5ae122d1ad641f P 3cf97c97242e411a90d80ef88e985a9a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbfd45351bfd41879acc3a7d6073fc55" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 38161 } } peers { permanent_uuid: "3cf97c97242e411a90d80ef88e985a9a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 32807 } } peers { permanent_uuid: "09c6b0f091c6462f9144d47f2ba2b118" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43043 } }
I20260811 18:59:59.965286 15501 leader_election.cc:288] T d886abbc036e4c438a5ae122d1ad641f P 3cf97c97242e411a90d80ef88e985a9a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bbfd45351bfd41879acc3a7d6073fc55 (127.13.109.1:38161), 09c6b0f091c6462f9144d47f2ba2b118 (127.13.109.3:43043)
I20260811 18:59:59.970459 15493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:59.966259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.2:33005 (local address 127.13.109.3:43043)
0811 18:59:59.966401 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:59.966406 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:59.966421 (+ 15us) server_negotiation.cc:407] Connection header received
0811 18:59:59.966509 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:59.966512 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:59.966565 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:59.966642 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:59:59.966921 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:59.967525 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:59.969178 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:59.969826 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:59.969950 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:59.970007 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:59.970256 (+ 249us) server_negotiation.cc:299] Negotiation successful
0811 18:59:59.970328 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 18:59:59.970484 15527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:59.966087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.3:43043 (local address 127.13.109.2:33005)
0811 18:59:59.966330 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:59.966351 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 18:59:59.966430 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:59:59.966663 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:59:59.966665 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:59:59.966702 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:59:59.966803 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:59.966807 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:59.967633 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:59.967636 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:59.969034 (+ 1398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:59.969046 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:59.969980 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:59.969983 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:59.970092 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:59.970156 (+ 64us) client_negotiation.cc:769] Sending connection context
0811 18:59:59.970237 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 18:59:59.970320 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"mutex_wait_us":46,"spinlock_wait_cycles":3328,"thread_start_us":78,"threads_started":1}
I20260811 18:59:59.970587 15489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:59.965840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.2:55925 (local address 127.13.109.1:38161)
0811 18:59:59.965994 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:59.965999 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:59.966161 (+ 162us) server_negotiation.cc:407] Connection header received
0811 18:59:59.966343 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:59.966346 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:59.966405 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:59.966460 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:59:59.966958 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:59.967446 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:59.969178 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:59.969826 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:59.969950 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:59.970007 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:59.970361 (+ 354us) server_negotiation.cc:299] Negotiation successful
0811 18:59:59.970460 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 18:59:59.971009 15526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:59.965618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.1:38161 (local address 127.13.109.2:55925)
0811 18:59:59.966016 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:59.966033 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 18:59:59.966187 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:59:59.966620 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:59:59.966625 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:59:59.966647 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:59:59.966811 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:59.966817 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:59.967566 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:59.967570 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:59.969034 (+ 1464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:59.969046 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:59.969946 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:59.969952 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:59.970066 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:59.970217 (+ 151us) client_negotiation.cc:769] Sending connection context
0811 18:59:59.970774 (+ 557us) client_negotiation.cc:240] Negotiation successful
0811 18:59:59.970868 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"mutex_wait_us":204,"spinlock_wait_cycles":26496,"thread_start_us":180,"threads_started":1}
I20260811 18:59:59.971097 15421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d886abbc036e4c438a5ae122d1ad641f" candidate_uuid: "3cf97c97242e411a90d80ef88e985a9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09c6b0f091c6462f9144d47f2ba2b118" is_pre_election: true
I20260811 18:59:59.971300 15421 raft_consensus.cc:2454] T d886abbc036e4c438a5ae122d1ad641f P 09c6b0f091c6462f9144d47f2ba2b118 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3cf97c97242e411a90d80ef88e985a9a in term 0.
I20260811 18:59:59.971609 15117 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d886abbc036e4c438a5ae122d1ad641f" candidate_uuid: "3cf97c97242e411a90d80ef88e985a9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbfd45351bfd41879acc3a7d6073fc55" is_pre_election: true
I20260811 18:59:59.971657 15201 leader_election.cc:302] T d886abbc036e4c438a5ae122d1ad641f P 3cf97c97242e411a90d80ef88e985a9a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09c6b0f091c6462f9144d47f2ba2b118, 3cf97c97242e411a90d80ef88e985a9a; no voters:
I20260811 18:59:59.971861 15117 raft_consensus.cc:2454] T d886abbc036e4c438a5ae122d1ad641f P bbfd45351bfd41879acc3a7d6073fc55 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3cf97c97242e411a90d80ef88e985a9a in term 0.
I20260811 18:59:59.971959 15501 raft_consensus.cc:2781] T d886abbc036e4c438a5ae122d1ad641f P 3cf97c97242e411a90d80ef88e985a9a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:59:59.972025 15501 raft_consensus.cc:482] T d886abbc036e4c438a5ae122d1ad641f P 3cf97c97242e411a90d80ef88e985a9a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:59:59.972074 15501 raft_consensus.cc:3037] T d886abbc036e4c438a5ae122d1ad641f P 3cf97c97242e411a90d80ef88e985a9a [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:59:59.973167 15501 raft_consensus.cc:504] T d886abbc036e4c438a5ae122d1ad641f P 3cf97c97242e411a90d80ef88e985a9a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbfd45351bfd41879acc3a7d6073fc55" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 38161 } } peers { permanent_uuid: "3cf97c97242e411a90d80ef88e985a9a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 32807 } } peers { permanent_uuid: "09c6b0f091c6462f9144d47f2ba2b118" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43043 } }
I20260811 18:59:59.973582 15501 leader_election.cc:288] T d886abbc036e4c438a5ae122d1ad641f P 3cf97c97242e411a90d80ef88e985a9a [CANDIDATE]: Term 1 election: Requested vote from peers bbfd45351bfd41879acc3a7d6073fc55 (127.13.109.1:38161), 09c6b0f091c6462f9144d47f2ba2b118 (127.13.109.3:43043)
I20260811 18:59:59.973978 15117 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d886abbc036e4c438a5ae122d1ad641f" candidate_uuid: "3cf97c97242e411a90d80ef88e985a9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbfd45351bfd41879acc3a7d6073fc55"
I20260811 18:59:59.974004 15421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d886abbc036e4c438a5ae122d1ad641f" candidate_uuid: "3cf97c97242e411a90d80ef88e985a9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09c6b0f091c6462f9144d47f2ba2b118"
I20260811 18:59:59.974092 15117 raft_consensus.cc:3037] T d886abbc036e4c438a5ae122d1ad641f P bbfd45351bfd41879acc3a7d6073fc55 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:59:59.974136 15421 raft_consensus.cc:3037] T d886abbc036e4c438a5ae122d1ad641f P 09c6b0f091c6462f9144d47f2ba2b118 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:59:59.975132 15421 raft_consensus.cc:2454] T d886abbc036e4c438a5ae122d1ad641f P 09c6b0f091c6462f9144d47f2ba2b118 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3cf97c97242e411a90d80ef88e985a9a in term 1.
I20260811 18:59:59.975150 15117 raft_consensus.cc:2454] T d886abbc036e4c438a5ae122d1ad641f P bbfd45351bfd41879acc3a7d6073fc55 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3cf97c97242e411a90d80ef88e985a9a in term 1.
I20260811 18:59:59.975461 15201 leader_election.cc:302] T d886abbc036e4c438a5ae122d1ad641f P 3cf97c97242e411a90d80ef88e985a9a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09c6b0f091c6462f9144d47f2ba2b118, 3cf97c97242e411a90d80ef88e985a9a; no voters:
I20260811 18:59:59.975672 15501 raft_consensus.cc:2781] T d886abbc036e4c438a5ae122d1ad641f P 3cf97c97242e411a90d80ef88e985a9a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:59:59.975870 15501 raft_consensus.cc:686] T d886abbc036e4c438a5ae122d1ad641f P 3cf97c97242e411a90d80ef88e985a9a [term 1 LEADER]: Becoming Leader. State: Replica: 3cf97c97242e411a90d80ef88e985a9a, State: Running, Role: LEADER
I20260811 18:59:59.976141 15501 consensus_queue.cc:237] T d886abbc036e4c438a5ae122d1ad641f P 3cf97c97242e411a90d80ef88e985a9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbfd45351bfd41879acc3a7d6073fc55" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 38161 } } peers { permanent_uuid: "3cf97c97242e411a90d80ef88e985a9a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 32807 } } peers { permanent_uuid: "09c6b0f091c6462f9144d47f2ba2b118" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43043 } }
I20260811 18:59:59.978752 14953 catalog_manager.cc:5495] T d886abbc036e4c438a5ae122d1ad641f P 3cf97c97242e411a90d80ef88e985a9a reported cstate change: term changed from 0 to 1, leader changed from <none> to 3cf97c97242e411a90d80ef88e985a9a (127.13.109.2). New cstate: current_term: 1 leader_uuid: "3cf97c97242e411a90d80ef88e985a9a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbfd45351bfd41879acc3a7d6073fc55" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 38161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3cf97c97242e411a90d80ef88e985a9a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 32807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09c6b0f091c6462f9144d47f2ba2b118" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43043 } health_report { overall_health: UNKNOWN } } }
Aug 11 19:00:00 dist-test-slave-0kls krb5kdc[14905](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474798, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.62@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
I20260811 19:00:00.041182 15536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:00.031866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:43061 (local address 127.0.0.1:41628)
0811 19:00:00.032368 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:00.032381 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:00.032535 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:00.032821 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:00.032824 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:00.033303 (+ 479us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:00.033404 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:00.033410 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:00.034273 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:00.034277 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:00.034842 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:00.034850 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:00.035320 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:00.035323 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:00.035398 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:00.035985 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:00.035997 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:00.037496 (+ 1499us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:00:00.039405 (+ 1909us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:00.039416 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:00.039422 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:00.039699 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:00.039979 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:00.039981 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:00.039983 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:00.040066 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:00.040552 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:00:00.040559 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:00:00.040688 (+ 129us) client_negotiation.cc:769] Sending connection context
0811 19:00:00.040847 (+ 159us) client_negotiation.cc:240] Negotiation successful
0811 19:00:00.041004 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":136,"threads_started":1}
I20260811 19:00:00.041203 15005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:00.031996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41628 (local address 127.13.109.62:43061)
0811 19:00:00.032246 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:00.032252 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:00.032472 (+ 220us) server_negotiation.cc:407] Connection header received
0811 19:00:00.032663 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:00.032672 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:00.032712 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:00.032801 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:00.033545 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:00.034149 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:00.034962 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:00.035201 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:00.035313 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:00.037734 (+ 2421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:00.037738 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:00.037740 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:00.037764 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:00.039287 (+ 1523us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:00.039809 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:00.039812 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:00.039813 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:00.039893 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:00.040159 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:00.040161 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:00.040163 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:00.040438 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:00.040527 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:00.040912 (+ 385us) server_negotiation.cc:299] Negotiation successful
0811 19:00:00.041069 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":34,"server-negotiator.queue_time_us":114}
I20260811 19:00:00.046334 14953 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.13.109.62@KRBTEST.COM'} at 127.0.0.1:41628:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 19:00:00.047181 14953 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 19:00:00.052372 15097 tablet_service.cc:1467] Processing CreateTablet for tablet f8560100cb704d0dbfdf2c37fe559b3f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c16f930689064e0fae0715c4687628a5]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 19:00:00.052440 15398 tablet_service.cc:1467] Processing CreateTablet for tablet f8560100cb704d0dbfdf2c37fe559b3f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c16f930689064e0fae0715c4687628a5]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 19:00:00.052393 15251 tablet_service.cc:1467] Processing CreateTablet for tablet f8560100cb704d0dbfdf2c37fe559b3f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c16f930689064e0fae0715c4687628a5]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 19:00:00.052858 15097 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f8560100cb704d0dbfdf2c37fe559b3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:00:00.052865 15251 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f8560100cb704d0dbfdf2c37fe559b3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:00:00.052865 15398 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f8560100cb704d0dbfdf2c37fe559b3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:00:00.055706 15495 tablet_bootstrap.cc:492] T f8560100cb704d0dbfdf2c37fe559b3f P bbfd45351bfd41879acc3a7d6073fc55: Bootstrap starting.
I20260811 19:00:00.055787 15497 tablet_bootstrap.cc:492] T f8560100cb704d0dbfdf2c37fe559b3f P 09c6b0f091c6462f9144d47f2ba2b118: Bootstrap starting.
I20260811 19:00:00.056439 15494 tablet_bootstrap.cc:492] T f8560100cb704d0dbfdf2c37fe559b3f P 3cf97c97242e411a90d80ef88e985a9a: Bootstrap starting.
I20260811 19:00:00.056638 15495 tablet_bootstrap.cc:654] T f8560100cb704d0dbfdf2c37fe559b3f P bbfd45351bfd41879acc3a7d6073fc55: Neither blocks nor log segments found. Creating new log.
I20260811 19:00:00.056845 15497 tablet_bootstrap.cc:654] T f8560100cb704d0dbfdf2c37fe559b3f P 09c6b0f091c6462f9144d47f2ba2b118: Neither blocks nor log segments found. Creating new log.
I20260811 19:00:00.057591 15494 tablet_bootstrap.cc:654] T f8560100cb704d0dbfdf2c37fe559b3f P 3cf97c97242e411a90d80ef88e985a9a: Neither blocks nor log segments found. Creating new log.
I20260811 19:00:00.058105 15495 tablet_bootstrap.cc:492] T f8560100cb704d0dbfdf2c37fe559b3f P bbfd45351bfd41879acc3a7d6073fc55: No bootstrap required, opened a new log
I20260811 19:00:00.058207 15495 ts_tablet_manager.cc:1397] T f8560100cb704d0dbfdf2c37fe559b3f P bbfd45351bfd41879acc3a7d6073fc55: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260811 19:00:00.058409 15497 tablet_bootstrap.cc:492] T f8560100cb704d0dbfdf2c37fe559b3f P 09c6b0f091c6462f9144d47f2ba2b118: No bootstrap required, opened a new log
I20260811 19:00:00.058482 15497 ts_tablet_manager.cc:1397] T f8560100cb704d0dbfdf2c37fe559b3f P 09c6b0f091c6462f9144d47f2ba2b118: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 19:00:00.058645 15495 raft_consensus.cc:348] T f8560100cb704d0dbfdf2c37fe559b3f P bbfd45351bfd41879acc3a7d6073fc55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbfd45351bfd41879acc3a7d6073fc55" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 38161 } } peers { permanent_uuid: "3cf97c97242e411a90d80ef88e985a9a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 32807 } } peers { permanent_uuid: "09c6b0f091c6462f9144d47f2ba2b118" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43043 } }
I20260811 19:00:00.058756 15495 raft_consensus.cc:374] T f8560100cb704d0dbfdf2c37fe559b3f P bbfd45351bfd41879acc3a7d6073fc55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:00:00.058790 15495 raft_consensus.cc:729] T f8560100cb704d0dbfdf2c37fe559b3f P bbfd45351bfd41879acc3a7d6073fc55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbfd45351bfd41879acc3a7d6073fc55, State: Initialized, Role: FOLLOWER
I20260811 19:00:00.058934 15495 consensus_queue.cc:260] T f8560100cb704d0dbfdf2c37fe559b3f P bbfd45351bfd41879acc3a7d6073fc55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbfd45351bfd41879acc3a7d6073fc55" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 38161 } } peers { permanent_uuid: "3cf97c97242e411a90d80ef88e985a9a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 32807 } } peers { permanent_uuid: "09c6b0f091c6462f9144d47f2ba2b118" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43043 } }
I20260811 19:00:00.058945 15497 raft_consensus.cc:348] T f8560100cb704d0dbfdf2c37fe559b3f P 09c6b0f091c6462f9144d47f2ba2b118 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbfd45351bfd41879acc3a7d6073fc55" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 38161 } } peers { permanent_uuid: "3cf97c97242e411a90d80ef88e985a9a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 32807 } } peers { permanent_uuid: "09c6b0f091c6462f9144d47f2ba2b118" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43043 } }
I20260811 19:00:00.059055 15497 raft_consensus.cc:374] T f8560100cb704d0dbfdf2c37fe559b3f P 09c6b0f091c6462f9144d47f2ba2b118 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:00:00.059093 15497 raft_consensus.cc:729] T f8560100cb704d0dbfdf2c37fe559b3f P 09c6b0f091c6462f9144d47f2ba2b118 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09c6b0f091c6462f9144d47f2ba2b118, State: Initialized, Role: FOLLOWER
I20260811 19:00:00.059171 15494 tablet_bootstrap.cc:492] T f8560100cb704d0dbfdf2c37fe559b3f P 3cf97c97242e411a90d80ef88e985a9a: No bootstrap required, opened a new log
I20260811 19:00:00.059257 15494 ts_tablet_manager.cc:1397] T f8560100cb704d0dbfdf2c37fe559b3f P 3cf97c97242e411a90d80ef88e985a9a: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 19:00:00.059209 15497 consensus_queue.cc:260] T f8560100cb704d0dbfdf2c37fe559b3f P 09c6b0f091c6462f9144d47f2ba2b118 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbfd45351bfd41879acc3a7d6073fc55" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 38161 } } peers { permanent_uuid: "3cf97c97242e411a90d80ef88e985a9a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 32807 } } peers { permanent_uuid: "09c6b0f091c6462f9144d47f2ba2b118" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43043 } }
I20260811 19:00:00.059435 15514 tablet_replica.cc:439] T f8560100cb704d0dbfdf2c37fe559b3f P bbfd45351bfd41879acc3a7d6073fc55: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbfd45351bfd41879acc3a7d6073fc55" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 38161 } } peers { permanent_uuid: "3cf97c97242e411a90d80ef88e985a9a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 32807 } } peers { permanent_uuid: "09c6b0f091c6462f9144d47f2ba2b118" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43043 } } }
I20260811 19:00:00.059599 15514 tablet_replica.cc:442] T f8560100cb704d0dbfdf2c37fe559b3f P bbfd45351bfd41879acc3a7d6073fc55: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 19:00:00.059582 15497 ts_tablet_manager.cc:1428] T f8560100cb704d0dbfdf2c37fe559b3f P 09c6b0f091c6462f9144d47f2ba2b118: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 19:00:00.059767 15494 raft_consensus.cc:348] T f8560100cb704d0dbfdf2c37fe559b3f P 3cf97c97242e411a90d80ef88e985a9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbfd45351bfd41879acc3a7d6073fc55" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 38161 } } peers { permanent_uuid: "3cf97c97242e411a90d80ef88e985a9a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 32807 } } peers { permanent_uuid: "09c6b0f091c6462f9144d47f2ba2b118" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43043 } }
I20260811 19:00:00.059904 15494 raft_consensus.cc:374] T f8560100cb704d0dbfdf2c37fe559b3f P 3cf97c97242e411a90d80ef88e985a9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:00:00.059849 15502 tablet_replica.cc:439] T f8560100cb704d0dbfdf2c37fe559b3f P 09c6b0f091c6462f9144d47f2ba2b118: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbfd45351bfd41879acc3a7d6073fc55" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 38161 } } peers { permanent_uuid: "3cf97c97242e411a90d80ef88e985a9a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 32807 } } peers { permanent_uuid: "09c6b0f091c6462f9144d47f2ba2b118" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43043 } } }
I20260811 19:00:00.059940 15494 raft_consensus.cc:729] T f8560100cb704d0dbfdf2c37fe559b3f P 3cf97c97242e411a90d80ef88e985a9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3cf97c97242e411a90d80ef88e985a9a, State: Initialized, Role: FOLLOWER
I20260811 19:00:00.059991 15502 tablet_replica.cc:442] T f8560100cb704d0dbfdf2c37fe559b3f P 09c6b0f091c6462f9144d47f2ba2b118: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 19:00:00.060109 15494 consensus_queue.cc:260] T f8560100cb704d0dbfdf2c37fe559b3f P 3cf97c97242e411a90d80ef88e985a9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbfd45351bfd41879acc3a7d6073fc55" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 38161 } } peers { permanent_uuid: "3cf97c97242e411a90d80ef88e985a9a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 32807 } } peers { permanent_uuid: "09c6b0f091c6462f9144d47f2ba2b118" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43043 } }
I20260811 19:00:00.060402 15495 ts_tablet_manager.cc:1428] T f8560100cb704d0dbfdf2c37fe559b3f P bbfd45351bfd41879acc3a7d6073fc55: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260811 19:00:00.060334 15501 tablet_replica.cc:439] T f8560100cb704d0dbfdf2c37fe559b3f P 3cf97c97242e411a90d80ef88e985a9a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbfd45351bfd41879acc3a7d6073fc55" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 38161 } } peers { permanent_uuid: "3cf97c97242e411a90d80ef88e985a9a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 32807 } } peers { permanent_uuid: "09c6b0f091c6462f9144d47f2ba2b118" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43043 } } }
I20260811 19:00:00.060475 15501 tablet_replica.cc:442] T f8560100cb704d0dbfdf2c37fe559b3f P 3cf97c97242e411a90d80ef88e985a9a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 19:00:00.060688 15494 ts_tablet_manager.cc:1428] T f8560100cb704d0dbfdf2c37fe559b3f P 3cf97c97242e411a90d80ef88e985a9a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 19:00:00.337531 15502 raft_consensus.cc:482] T f8560100cb704d0dbfdf2c37fe559b3f P 09c6b0f091c6462f9144d47f2ba2b118 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:00:00.337694 15502 raft_consensus.cc:504] T f8560100cb704d0dbfdf2c37fe559b3f P 09c6b0f091c6462f9144d47f2ba2b118 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbfd45351bfd41879acc3a7d6073fc55" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 38161 } } peers { permanent_uuid: "3cf97c97242e411a90d80ef88e985a9a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 32807 } } peers { permanent_uuid: "09c6b0f091c6462f9144d47f2ba2b118" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43043 } }
I20260811 19:00:00.339314 15502 leader_election.cc:288] T f8560100cb704d0dbfdf2c37fe559b3f P 09c6b0f091c6462f9144d47f2ba2b118 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bbfd45351bfd41879acc3a7d6073fc55 (127.13.109.1:38161), 3cf97c97242e411a90d80ef88e985a9a (127.13.109.2:32807)
I20260811 19:00:00.344112 15541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:00.339694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.2:32807 (local address 127.13.109.3:35243)
0811 19:00:00.340237 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:00.340253 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:00.340399 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:00.340692 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:00.340695 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:00.340735 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:00:00.340839 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:00.340844 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:00.341728 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:00.341732 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:00.342824 (+ 1092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:00.342834 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:00.343598 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:00.343601 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:00.343720 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:00.343740 (+ 20us) client_negotiation.cc:769] Sending connection context
0811 19:00:00.343825 (+ 85us) client_negotiation.cc:240] Negotiation successful
0811 19:00:00.343934 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"mutex_wait_us":74,"spinlock_wait_cycles":768,"thread_start_us":73,"threads_started":1}
I20260811 19:00:00.344101 15542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:00.339851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.3:35243 (local address 127.13.109.2:32807)
0811 19:00:00.340143 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:00.340149 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:00.340351 (+ 202us) server_negotiation.cc:407] Connection header received
0811 19:00:00.340532 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:00.340539 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:00.340580 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:00.340671 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:00:00.340946 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:00.341620 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:00.342945 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:00.343464 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:00.343597 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:00.343653 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:00.343844 (+ 191us) server_negotiation.cc:299] Negotiation successful
0811 19:00:00.343929 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":103,"threads_started":1}
I20260811 19:00:00.344113 15489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:00.339848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.3:37043 (local address 127.13.109.1:38161)
0811 19:00:00.340002 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:00.340026 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:00.340242 (+ 216us) server_negotiation.cc:407] Connection header received
0811 19:00:00.340469 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:00.340473 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:00.340532 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:00.340640 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:00:00.340973 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:00.341704 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:00.342970 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:00.343422 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:00.343580 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:00.343644 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:00.343881 (+ 237us) server_negotiation.cc:299] Negotiation successful
0811 19:00:00.343970 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 19:00:00.344156 15540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:00.339694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.1:38161 (local address 127.13.109.3:37043)
0811 19:00:00.340124 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:00.340142 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:00.340322 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:00.340654 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:00.340657 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:00.340675 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:00:00.340845 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:00.340851 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:00.341855 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:00.341860 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:00.342828 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:00.342838 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:00.343568 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:00.343572 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:00.343733 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:00.343759 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 19:00:00.343861 (+ 102us) client_negotiation.cc:240] Negotiation successful
0811 19:00:00.343968 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"mutex_wait_us":47,"spinlock_wait_cycles":24192,"thread_start_us":85,"threads_started":1}
I20260811 19:00:00.344732 15117 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8560100cb704d0dbfdf2c37fe559b3f" candidate_uuid: "09c6b0f091c6462f9144d47f2ba2b118" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbfd45351bfd41879acc3a7d6073fc55" is_pre_election: true
I20260811 19:00:00.344920 15117 raft_consensus.cc:2454] T f8560100cb704d0dbfdf2c37fe559b3f P bbfd45351bfd41879acc3a7d6073fc55 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09c6b0f091c6462f9144d47f2ba2b118 in term 0.
I20260811 19:00:00.344884 15253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8560100cb704d0dbfdf2c37fe559b3f" candidate_uuid: "09c6b0f091c6462f9144d47f2ba2b118" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3cf97c97242e411a90d80ef88e985a9a" is_pre_election: true
I20260811 19:00:00.345067 15253 raft_consensus.cc:2454] T f8560100cb704d0dbfdf2c37fe559b3f P 3cf97c97242e411a90d80ef88e985a9a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09c6b0f091c6462f9144d47f2ba2b118 in term 0.
I20260811 19:00:00.345413 15353 leader_election.cc:302] T f8560100cb704d0dbfdf2c37fe559b3f P 09c6b0f091c6462f9144d47f2ba2b118 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 09c6b0f091c6462f9144d47f2ba2b118, 3cf97c97242e411a90d80ef88e985a9a, bbfd45351bfd41879acc3a7d6073fc55; no voters:
I20260811 19:00:00.345741 15502 raft_consensus.cc:2781] T f8560100cb704d0dbfdf2c37fe559b3f P 09c6b0f091c6462f9144d47f2ba2b118 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 19:00:00.345845 15502 raft_consensus.cc:482] T f8560100cb704d0dbfdf2c37fe559b3f P 09c6b0f091c6462f9144d47f2ba2b118 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 19:00:00.345903 15502 raft_consensus.cc:3037] T f8560100cb704d0dbfdf2c37fe559b3f P 09c6b0f091c6462f9144d47f2ba2b118 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:00:00.346951 15502 raft_consensus.cc:504] T f8560100cb704d0dbfdf2c37fe559b3f P 09c6b0f091c6462f9144d47f2ba2b118 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbfd45351bfd41879acc3a7d6073fc55" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 38161 } } peers { permanent_uuid: "3cf97c97242e411a90d80ef88e985a9a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 32807 } } peers { permanent_uuid: "09c6b0f091c6462f9144d47f2ba2b118" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43043 } }
I20260811 19:00:00.347314 15502 leader_election.cc:288] T f8560100cb704d0dbfdf2c37fe559b3f P 09c6b0f091c6462f9144d47f2ba2b118 [CANDIDATE]: Term 1 election: Requested vote from peers bbfd45351bfd41879acc3a7d6073fc55 (127.13.109.1:38161), 3cf97c97242e411a90d80ef88e985a9a (127.13.109.2:32807)
I20260811 19:00:00.347656 15253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8560100cb704d0dbfdf2c37fe559b3f" candidate_uuid: "09c6b0f091c6462f9144d47f2ba2b118" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3cf97c97242e411a90d80ef88e985a9a"
I20260811 19:00:00.347779 15253 raft_consensus.cc:3037] T f8560100cb704d0dbfdf2c37fe559b3f P 3cf97c97242e411a90d80ef88e985a9a [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:00:00.347848 15117 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8560100cb704d0dbfdf2c37fe559b3f" candidate_uuid: "09c6b0f091c6462f9144d47f2ba2b118" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbfd45351bfd41879acc3a7d6073fc55"
I20260811 19:00:00.347976 15117 raft_consensus.cc:3037] T f8560100cb704d0dbfdf2c37fe559b3f P bbfd45351bfd41879acc3a7d6073fc55 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:00:00.348716 15253 raft_consensus.cc:2454] T f8560100cb704d0dbfdf2c37fe559b3f P 3cf97c97242e411a90d80ef88e985a9a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09c6b0f091c6462f9144d47f2ba2b118 in term 1.
I20260811 19:00:00.348987 15117 raft_consensus.cc:2454] T f8560100cb704d0dbfdf2c37fe559b3f P bbfd45351bfd41879acc3a7d6073fc55 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09c6b0f091c6462f9144d47f2ba2b118 in term 1.
I20260811 19:00:00.349006 15350 leader_election.cc:302] T f8560100cb704d0dbfdf2c37fe559b3f P 09c6b0f091c6462f9144d47f2ba2b118 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09c6b0f091c6462f9144d47f2ba2b118, 3cf97c97242e411a90d80ef88e985a9a; no voters:
I20260811 19:00:00.349170 15502 raft_consensus.cc:2781] T f8560100cb704d0dbfdf2c37fe559b3f P 09c6b0f091c6462f9144d47f2ba2b118 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:00:00.349355 15502 raft_consensus.cc:686] T f8560100cb704d0dbfdf2c37fe559b3f P 09c6b0f091c6462f9144d47f2ba2b118 [term 1 LEADER]: Becoming Leader. State: Replica: 09c6b0f091c6462f9144d47f2ba2b118, State: Running, Role: LEADER
I20260811 19:00:00.349665 15502 consensus_queue.cc:237] T f8560100cb704d0dbfdf2c37fe559b3f P 09c6b0f091c6462f9144d47f2ba2b118 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbfd45351bfd41879acc3a7d6073fc55" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 38161 } } peers { permanent_uuid: "3cf97c97242e411a90d80ef88e985a9a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 32807 } } peers { permanent_uuid: "09c6b0f091c6462f9144d47f2ba2b118" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43043 } }
I20260811 19:00:00.351161 15543 tablet_replica.cc:439] T f8560100cb704d0dbfdf2c37fe559b3f P 09c6b0f091c6462f9144d47f2ba2b118: TxnStatusTablet state changed. Reason: New leader 09c6b0f091c6462f9144d47f2ba2b118. Latest consensus state: current_term: 1 leader_uuid: "09c6b0f091c6462f9144d47f2ba2b118" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbfd45351bfd41879acc3a7d6073fc55" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 38161 } } peers { permanent_uuid: "3cf97c97242e411a90d80ef88e985a9a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 32807 } } peers { permanent_uuid: "09c6b0f091c6462f9144d47f2ba2b118" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43043 } } }
I20260811 19:00:00.351301 15543 tablet_replica.cc:442] T f8560100cb704d0dbfdf2c37fe559b3f P 09c6b0f091c6462f9144d47f2ba2b118: This TxnStatusTablet replica's current role is: LEADER
I20260811 19:00:00.352618 14952 catalog_manager.cc:5495] T f8560100cb704d0dbfdf2c37fe559b3f P 09c6b0f091c6462f9144d47f2ba2b118 reported cstate change: term changed from 0 to 1, leader changed from <none> to 09c6b0f091c6462f9144d47f2ba2b118 (127.13.109.3). New cstate: current_term: 1 leader_uuid: "09c6b0f091c6462f9144d47f2ba2b118" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbfd45351bfd41879acc3a7d6073fc55" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 38161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3cf97c97242e411a90d80ef88e985a9a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 32807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09c6b0f091c6462f9144d47f2ba2b118" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43043 } health_report { overall_health: HEALTHY } } }
I20260811 19:00:00.358032 15545 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 19:00:00.358297 15545 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 19:00:00.371182 15536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:00.367252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.3:43043 (local address 127.0.0.1:50852)
0811 19:00:00.367438 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:00.367459 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:00.367591 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:00.367962 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:00.367965 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:00.367973 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 19:00:00.368087 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:00.368093 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:00.368830 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:00.368834 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:00.369488 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:00.369500 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:00.370024 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:00.370028 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:00.370122 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:00.370171 (+ 49us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 19:00:00.370893 (+ 722us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 19:00:00.370911 (+ 18us) client_negotiation.cc:769] Sending connection context
0811 19:00:00.370979 (+ 68us) client_negotiation.cc:240] Negotiation successful
0811 19:00:00.371066 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260811 19:00:00.371857 15493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:00.367327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50852 (local address 127.13.109.3:43043)
0811 19:00:00.367514 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:00.367518 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:00.367532 (+ 14us) server_negotiation.cc:407] Connection header received
0811 19:00:00.367754 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:00.367756 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:00.367810 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:00.367936 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:00:00.368198 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:00.368716 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:00.369611 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:00.369910 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:00.370023 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:00.370324 (+ 301us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:00:00.370793 (+ 469us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:00:00.370915 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:00.371700 (+ 785us) server_negotiation.cc:299] Negotiation successful
0811 19:00:00.371750 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111}
I20260811 19:00:00.374946 15253 raft_consensus.cc:1261] T f8560100cb704d0dbfdf2c37fe559b3f P 3cf97c97242e411a90d80ef88e985a9a [term 1 FOLLOWER]: Refusing update from remote peer 09c6b0f091c6462f9144d47f2ba2b118: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:00:00.375183 15117 raft_consensus.cc:1261] T f8560100cb704d0dbfdf2c37fe559b3f P bbfd45351bfd41879acc3a7d6073fc55 [term 1 FOLLOWER]: Refusing update from remote peer 09c6b0f091c6462f9144d47f2ba2b118: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:00:00.375588 15502 consensus_queue.cc:1035] T f8560100cb704d0dbfdf2c37fe559b3f P 09c6b0f091c6462f9144d47f2ba2b118 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3cf97c97242e411a90d80ef88e985a9a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 32807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:00:00.375765 15543 consensus_queue.cc:1035] T f8560100cb704d0dbfdf2c37fe559b3f P 09c6b0f091c6462f9144d47f2ba2b118 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bbfd45351bfd41879acc3a7d6073fc55" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 38161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:00:00.377285 15501 tablet_replica.cc:439] T f8560100cb704d0dbfdf2c37fe559b3f P 3cf97c97242e411a90d80ef88e985a9a: TxnStatusTablet state changed. Reason: New leader 09c6b0f091c6462f9144d47f2ba2b118. Latest consensus state: current_term: 1 leader_uuid: "09c6b0f091c6462f9144d47f2ba2b118" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbfd45351bfd41879acc3a7d6073fc55" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 38161 } } peers { permanent_uuid: "3cf97c97242e411a90d80ef88e985a9a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 32807 } } peers { permanent_uuid: "09c6b0f091c6462f9144d47f2ba2b118" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43043 } } }
I20260811 19:00:00.377432 15501 tablet_replica.cc:442] T f8560100cb704d0dbfdf2c37fe559b3f P 3cf97c97242e411a90d80ef88e985a9a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 19:00:00.377604 15514 tablet_replica.cc:439] T f8560100cb704d0dbfdf2c37fe559b3f P bbfd45351bfd41879acc3a7d6073fc55: TxnStatusTablet state changed. Reason: New leader 09c6b0f091c6462f9144d47f2ba2b118. Latest consensus state: current_term: 1 leader_uuid: "09c6b0f091c6462f9144d47f2ba2b118" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbfd45351bfd41879acc3a7d6073fc55" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 38161 } } peers { permanent_uuid: "3cf97c97242e411a90d80ef88e985a9a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 32807 } } peers { permanent_uuid: "09c6b0f091c6462f9144d47f2ba2b118" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43043 } } }
I20260811 19:00:00.377718 15514 tablet_replica.cc:442] T f8560100cb704d0dbfdf2c37fe559b3f P bbfd45351bfd41879acc3a7d6073fc55: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 19:00:00.381036 15502 tablet_replica.cc:439] T f8560100cb704d0dbfdf2c37fe559b3f P 09c6b0f091c6462f9144d47f2ba2b118: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "09c6b0f091c6462f9144d47f2ba2b118" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbfd45351bfd41879acc3a7d6073fc55" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 38161 } } peers { permanent_uuid: "3cf97c97242e411a90d80ef88e985a9a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 32807 } } peers { permanent_uuid: "09c6b0f091c6462f9144d47f2ba2b118" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43043 } } }
I20260811 19:00:00.381155 15502 tablet_replica.cc:442] T f8560100cb704d0dbfdf2c37fe559b3f P 09c6b0f091c6462f9144d47f2ba2b118: This TxnStatusTablet replica's current role is: LEADER
I20260811 19:00:00.381153 15514 tablet_replica.cc:439] T f8560100cb704d0dbfdf2c37fe559b3f P bbfd45351bfd41879acc3a7d6073fc55: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "09c6b0f091c6462f9144d47f2ba2b118" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbfd45351bfd41879acc3a7d6073fc55" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 38161 } } peers { permanent_uuid: "3cf97c97242e411a90d80ef88e985a9a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 32807 } } peers { permanent_uuid: "09c6b0f091c6462f9144d47f2ba2b118" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43043 } } }
I20260811 19:00:00.381245 15514 tablet_replica.cc:442] T f8560100cb704d0dbfdf2c37fe559b3f P bbfd45351bfd41879acc3a7d6073fc55: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 19:00:00.381372 15551 mvcc.cc:205] Tried to move back new op lower bound from 7317400782330146816 to 7317400782235164672. Current Snapshot: MvccSnapshot[applied={T|T < 7317400782330146816}]
I20260811 19:00:00.381379 15501 tablet_replica.cc:439] T f8560100cb704d0dbfdf2c37fe559b3f P 3cf97c97242e411a90d80ef88e985a9a: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "09c6b0f091c6462f9144d47f2ba2b118" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbfd45351bfd41879acc3a7d6073fc55" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 38161 } } peers { permanent_uuid: "3cf97c97242e411a90d80ef88e985a9a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 32807 } } peers { permanent_uuid: "09c6b0f091c6462f9144d47f2ba2b118" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43043 } } }
I20260811 19:00:00.381470 15501 tablet_replica.cc:442] T f8560100cb704d0dbfdf2c37fe559b3f P 3cf97c97242e411a90d80ef88e985a9a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 19:00:00.382138 15554 tablet_replica.cc:439] T f8560100cb704d0dbfdf2c37fe559b3f P 09c6b0f091c6462f9144d47f2ba2b118: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "09c6b0f091c6462f9144d47f2ba2b118" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbfd45351bfd41879acc3a7d6073fc55" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 38161 } } peers { permanent_uuid: "3cf97c97242e411a90d80ef88e985a9a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 32807 } } peers { permanent_uuid: "09c6b0f091c6462f9144d47f2ba2b118" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43043 } } }
I20260811 19:00:00.382285 15554 tablet_replica.cc:442] T f8560100cb704d0dbfdf2c37fe559b3f P 09c6b0f091c6462f9144d47f2ba2b118: This TxnStatusTablet replica's current role is: LEADER
I20260811 19:00:00.384267 15549 mvcc.cc:205] Tried to move back new op lower bound from 7317400782330146816 to 7317400782235164672. Current Snapshot: MvccSnapshot[applied={T|T < 7317400782330146816 or (T in {7317400782330146816})}]
I20260811 19:00:00.399750 15542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:00.395036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41226 (local address 127.13.109.2:32807)
0811 19:00:00.395168 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:00.395174 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:00.395495 (+ 321us) server_negotiation.cc:407] Connection header received
0811 19:00:00.395662 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:00.395665 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:00.395726 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:00.395819 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:00:00.396138 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:00.396823 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:00.398181 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:00.398391 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:00.398475 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:00.398800 (+ 325us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:00:00.399302 (+ 502us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:00:00.399393 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:00.399511 (+ 118us) server_negotiation.cc:299] Negotiation successful
0811 19:00:00.399568 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260811 19:00:00.412097 15493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:00.407765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.2:56137 (local address 127.13.109.3:43043)
0811 19:00:00.407914 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:00.407919 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:00.408062 (+ 143us) server_negotiation.cc:407] Connection header received
0811 19:00:00.408153 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:00.408157 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:00.408216 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:00.408306 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:00:00.408556 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:00.409428 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:00.410769 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:00.411154 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:00.411274 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:00.411629 (+ 355us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:00:00.411717 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:00:00.411802 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:00.411894 (+ 92us) server_negotiation.cc:299] Negotiation successful
0811 19:00:00.411951 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 19:00:00.412120 15562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:00.407654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.3:43043 (local address 127.13.109.2:56137)
0811 19:00:00.407968 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:00.407981 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:00.408071 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:00.408320 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:00.408322 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:00.408328 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 19:00:00.408431 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:00.408435 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:00.409542 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:00.409545 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:00.410629 (+ 1084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:00.410640 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:00.411342 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:00.411345 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:00.411428 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:00.411471 (+ 43us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 19:00:00.411806 (+ 335us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 19:00:00.411811 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 19:00:00.411861 (+ 50us) client_negotiation.cc:240] Negotiation successful
0811 19:00:00.411925 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":118,"threads_started":1}
I20260811 19:00:00.419647 15421 raft_consensus.cc:1261] T d886abbc036e4c438a5ae122d1ad641f P 09c6b0f091c6462f9144d47f2ba2b118 [term 1 FOLLOWER]: Refusing update from remote peer 3cf97c97242e411a90d80ef88e985a9a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:00:00.419718 15117 raft_consensus.cc:1261] T d886abbc036e4c438a5ae122d1ad641f P bbfd45351bfd41879acc3a7d6073fc55 [term 1 FOLLOWER]: Refusing update from remote peer 3cf97c97242e411a90d80ef88e985a9a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:00:00.420166 15529 consensus_queue.cc:1035] T d886abbc036e4c438a5ae122d1ad641f P 3cf97c97242e411a90d80ef88e985a9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "09c6b0f091c6462f9144d47f2ba2b118" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:00:00.420367 15501 consensus_queue.cc:1035] T d886abbc036e4c438a5ae122d1ad641f P 3cf97c97242e411a90d80ef88e985a9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "bbfd45351bfd41879acc3a7d6073fc55" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 38161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:00:00.447535 15542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:00.443591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.3:60813 (local address 127.13.109.2:32807)
0811 19:00:00.443719 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:00.443724 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:00.443966 (+ 242us) server_negotiation.cc:407] Connection header received
0811 19:00:00.444101 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:00.444104 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:00.444146 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:00.444223 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:00:00.444555 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:00.445196 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:00.446421 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:00.446649 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:00.446766 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:00.447076 (+ 310us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:00:00.447159 (+ 83us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:00:00.447247 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:00.447339 (+ 92us) server_negotiation.cc:299] Negotiation successful
0811 19:00:00.447389 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 19:00:00.447544 15569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:00.443479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.2:32807 (local address 127.13.109.3:60813)
0811 19:00:00.443867 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:00.443884 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:00.444001 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:00.444262 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:00.444265 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:00.444287 (+ 22us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 19:00:00.444434 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:00.444440 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:00.445310 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:00.445316 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:00.446299 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:00.446309 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:00.446785 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:00.446789 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:00.446889 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:00.446935 (+ 46us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 19:00:00.447255 (+ 320us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 19:00:00.447265 (+ 10us) client_negotiation.cc:769] Sending connection context
0811 19:00:00.447329 (+ 64us) client_negotiation.cc:240] Negotiation successful
0811 19:00:00.447406 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":113,"threads_started":1}
I20260811 19:00:00.475934 14952 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41614:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:00:00.476140 14952 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41614:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:00:00.478906 14952 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1f76e5475fab4ce384e2d95956b9aa32: Sending DeleteTablet for 3 replicas of tablet d886abbc036e4c438a5ae122d1ad641f
I20260811 19:00:00.479867 15251 tablet_service.cc:1514] Processing DeleteTablet for tablet d886abbc036e4c438a5ae122d1ad641f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:00:00 UTC) from {username='kudu', principal='kudu/127.13.109.62@KRBTEST.COM'} at 127.0.0.1:41214
I20260811 19:00:00.479943 15097 tablet_service.cc:1514] Processing DeleteTablet for tablet d886abbc036e4c438a5ae122d1ad641f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:00:00 UTC) from {username='kudu', principal='kudu/127.13.109.62@KRBTEST.COM'} at 127.0.0.1:47876
I20260811 19:00:00.480357 15398 tablet_service.cc:1514] Processing DeleteTablet for tablet d886abbc036e4c438a5ae122d1ad641f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:00:00 UTC) from {username='kudu', principal='kudu/127.13.109.62@KRBTEST.COM'} at 127.0.0.1:50838
I20260811 19:00:00.480800 15574 tablet_replica.cc:330] T d886abbc036e4c438a5ae122d1ad641f P 09c6b0f091c6462f9144d47f2ba2b118: stopping tablet replica
I20260811 19:00:00.481014 15574 raft_consensus.cc:2229] T d886abbc036e4c438a5ae122d1ad641f P 09c6b0f091c6462f9144d47f2ba2b118 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:00:00.481150 15574 raft_consensus.cc:2258] T d886abbc036e4c438a5ae122d1ad641f P 09c6b0f091c6462f9144d47f2ba2b118 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:00:00.482125 15574 ts_tablet_manager.cc:1905] T d886abbc036e4c438a5ae122d1ad641f P 09c6b0f091c6462f9144d47f2ba2b118: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:00:00.483259 15572 tablet_replica.cc:330] T d886abbc036e4c438a5ae122d1ad641f P 3cf97c97242e411a90d80ef88e985a9a: stopping tablet replica
I20260811 19:00:00.483881 15572 raft_consensus.cc:2229] T d886abbc036e4c438a5ae122d1ad641f P 3cf97c97242e411a90d80ef88e985a9a [term 1 LEADER]: Raft consensus shutting down.
I20260811 19:00:00.484421 15572 raft_consensus.cc:2258] T d886abbc036e4c438a5ae122d1ad641f P 3cf97c97242e411a90d80ef88e985a9a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:00:00.485296 15572 ts_tablet_manager.cc:1905] T d886abbc036e4c438a5ae122d1ad641f P 3cf97c97242e411a90d80ef88e985a9a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:00:00.485651 15574 ts_tablet_manager.cc:1918] T d886abbc036e4c438a5ae122d1ad641f P 09c6b0f091c6462f9144d47f2ba2b118: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 19:00:00.485749 15574 log.cc:1196] T d886abbc036e4c438a5ae122d1ad641f P 09c6b0f091c6462f9144d47f2ba2b118: Deleting WAL directory at /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/minicluster-data/ts-2/wal/wals/d886abbc036e4c438a5ae122d1ad641f
I20260811 19:00:00.485963 15573 tablet_replica.cc:330] T d886abbc036e4c438a5ae122d1ad641f P bbfd45351bfd41879acc3a7d6073fc55: stopping tablet replica
I20260811 19:00:00.486079 15574 ts_tablet_manager.cc:1939] T d886abbc036e4c438a5ae122d1ad641f P 09c6b0f091c6462f9144d47f2ba2b118: Deleting consensus metadata
I20260811 19:00:00.486158 15573 raft_consensus.cc:2229] T d886abbc036e4c438a5ae122d1ad641f P bbfd45351bfd41879acc3a7d6073fc55 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:00:00.486304 15573 raft_consensus.cc:2258] T d886abbc036e4c438a5ae122d1ad641f P bbfd45351bfd41879acc3a7d6073fc55 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:00:00.486852 14939 catalog_manager.cc:4841] TS 09c6b0f091c6462f9144d47f2ba2b118 (127.13.109.3:43043): tablet d886abbc036e4c438a5ae122d1ad641f (table test-table [id=d4aae8c4958142d69688388dfa4523d5]) successfully deleted
I20260811 19:00:00.489578 15572 ts_tablet_manager.cc:1918] T d886abbc036e4c438a5ae122d1ad641f P 3cf97c97242e411a90d80ef88e985a9a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 19:00:00.489672 15572 log.cc:1196] T d886abbc036e4c438a5ae122d1ad641f P 3cf97c97242e411a90d80ef88e985a9a: Deleting WAL directory at /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/minicluster-data/ts-1/wal/wals/d886abbc036e4c438a5ae122d1ad641f
I20260811 19:00:00.490113 15572 ts_tablet_manager.cc:1939] T d886abbc036e4c438a5ae122d1ad641f P 3cf97c97242e411a90d80ef88e985a9a: Deleting consensus metadata
I20260811 19:00:00.490979 14937 catalog_manager.cc:4841] TS 3cf97c97242e411a90d80ef88e985a9a (127.13.109.2:32807): tablet d886abbc036e4c438a5ae122d1ad641f (table test-table [id=d4aae8c4958142d69688388dfa4523d5]) successfully deleted
I20260811 19:00:00.494360 15573 ts_tablet_manager.cc:1905] T d886abbc036e4c438a5ae122d1ad641f P bbfd45351bfd41879acc3a7d6073fc55: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:00:00.497586 15573 ts_tablet_manager.cc:1918] T d886abbc036e4c438a5ae122d1ad641f P bbfd45351bfd41879acc3a7d6073fc55: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 19:00:00.497723 15573 log.cc:1196] T d886abbc036e4c438a5ae122d1ad641f P bbfd45351bfd41879acc3a7d6073fc55: Deleting WAL directory at /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474792496492-13748-0/minicluster-data/ts-0/wal/wals/d886abbc036e4c438a5ae122d1ad641f
I20260811 19:00:00.498051 15573 ts_tablet_manager.cc:1939] T d886abbc036e4c438a5ae122d1ad641f P bbfd45351bfd41879acc3a7d6073fc55: Deleting consensus metadata
Aug 11 19:00:00 dist-test-slave-0kls krb5kdc[14905](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474799, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.13.109.1@KRBTEST.COM
I20260811 19:00:00.499526 14939 catalog_manager.cc:4841] TS bbfd45351bfd41879acc3a7d6073fc55 (127.13.109.1:38161): tablet d886abbc036e4c438a5ae122d1ad641f (table test-table [id=d4aae8c4958142d69688388dfa4523d5]) successfully deleted
I20260811 19:00:00.504446 15489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:00.483080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50030 (local address 127.13.109.1:38161)
0811 19:00:00.483215 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:00.483220 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:00.491101 (+ 7881us) server_negotiation.cc:407] Connection header received
0811 19:00:00.491372 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:00.491376 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:00.491434 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:00.491522 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:00.492484 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:00.493326 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:00.494713 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:00.495090 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:00.495182 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:00.500141 (+ 4959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:00.500152 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:00.500169 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:00.500199 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:00.502378 (+ 2179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:00.502941 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:00.502948 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:00.502950 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:00.503046 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:00.503373 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:00.503378 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:00.503380 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:00.503694 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:00.503891 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:00.504150 (+ 259us) server_negotiation.cc:299] Negotiation successful
0811 19:00:00.504281 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
W20260811 19:00:00.505290 15162 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50030
I20260811 19:00:00.517616 15005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:00.507301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54804 (local address 127.13.109.62:43061)
0811 19:00:00.507458 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:00.507462 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:00.507987 (+ 525us) server_negotiation.cc:407] Connection header received
0811 19:00:00.508084 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:00.508087 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:00.508169 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:00.508259 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:00.509051 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:00.510057 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:00.511221 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:00.511620 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:00.511741 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:00.513541 (+ 1800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:00.513546 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:00.513549 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:00.513580 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:00.515619 (+ 2039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:00.516127 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:00.516132 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:00.516134 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:00.516243 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:00.516567 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:00.516573 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:00.516576 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:00.516845 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:00.516995 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:00.517275 (+ 280us) server_negotiation.cc:299] Negotiation successful
0811 19:00:00.517413 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
W20260811 19:00:00.518299 14952 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54804
I20260811 19:00:00.519881 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 15029
I20260811 19:00:00.536139 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 15179
I20260811 19:00:00.546267 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 15333
I20260811 19:00:00.556979 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 14918
2026-08-11T19:00:00Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (3171 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:00:00 dist-test-slave-0kls krb5kdc[15582](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:00:00 dist-test-slave-0kls krb5kdc[15582](info): set up 2 sockets
Aug 11 19:00:00 dist-test-slave-0kls krb5kdc[15582](info): commencing operation
krb5kdc: starting...
W20260811 19:00:01.651156 13748 mini_kdc.cc:121] Time spent starting KDC: real 1.067s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/krb5kdc/test-user.keytab.
Aug 11 19:00:01 dist-test-slave-0kls krb5kdc[15582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474801, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:00:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:00:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.109.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.109.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.109.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab.
Entry for principal kudu/127.13.109.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab.
I20260811 19:00:01.796763 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.62:34491
--webserver_interface=127.13.109.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab
--principal=kudu/127.13.109.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:41169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.109.62:34491
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:00:01.919441 15595 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:00:01.919822 15595 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:00:01.919871 15595 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:00:01.924923 15595 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:00:01.925009 15595 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 19:00:01.925030 15595 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:00:01.925052 15595 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:00:01.925122 15595 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:00:01.925150 15595 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:00:01.931099 15595 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:41169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.109.62:34491
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab
--principal=kudu/127.13.109.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.62:34491
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.109.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15595
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:00:01.932394 15595 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:00:01.933542 15595 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:00:01.943004 15600 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:01.943017 15603 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:01.943017 15601 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:01.943423 15595 server_base.cc:1034] running on GCE node
I20260811 19:00:01.944124 15595 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:00:01.945405 15595 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:00:01.946583 15595 hybrid_clock.cc:648] HybridClock initialized: now 1786474801946533 us; error 65 us; skew 500 ppm
Aug 11 19:00:01 dist-test-slave-0kls krb5kdc[15582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474801, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:00:01.949710 15595 init.cc:377] Logged in from keytab as kudu/127.13.109.62@KRBTEST.COM (short username kudu)
I20260811 19:00:01.950886 15595 webserver.cc:460] Webserver started at http://127.13.109.62:42665/ using document root <none> and password file <none>
I20260811 19:00:01.951560 15595 fs_manager.cc:359] Metadata directory not provided
I20260811 19:00:01.951629 15595 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:00:01.951905 15595 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:00:01.954375 15595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/minicluster-data/master-0/data/instance:
uuid: "c5c0646e359b477aaf79adc6b3f50482"
format_stamp: "Formatted at 2026-08-11 19:00:01 on dist-test-slave-0kls"
server_key: "b3e50edecc29134ea574369bdf626c71"
server_key_iv: "1ed36839ef2ef370f0bceaa9f536bc72"
server_key_version: "encryptionkey@0"
I20260811 19:00:01.955061 15595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/minicluster-data/master-0/wal/instance:
uuid: "c5c0646e359b477aaf79adc6b3f50482"
format_stamp: "Formatted at 2026-08-11 19:00:01 on dist-test-slave-0kls"
server_key: "b3e50edecc29134ea574369bdf626c71"
server_key_iv: "1ed36839ef2ef370f0bceaa9f536bc72"
server_key_version: "encryptionkey@0"
I20260811 19:00:01.959659 15595 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 19:00:01.963042 15610 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:01.964428 15595 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 19:00:01.964613 15595 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/minicluster-data/master-0/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/minicluster-data/master-0/wal
uuid: "c5c0646e359b477aaf79adc6b3f50482"
format_stamp: "Formatted at 2026-08-11 19:00:01 on dist-test-slave-0kls"
server_key: "b3e50edecc29134ea574369bdf626c71"
server_key_iv: "1ed36839ef2ef370f0bceaa9f536bc72"
server_key_version: "encryptionkey@0"
I20260811 19:00:01.964764 15595 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:00:01.974879 15595 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:00:01.978765 15595 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:00:01.979009 15595 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:00:01.987913 15595 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.62:34491
I20260811 19:00:01.987939 15676 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.62:34491 every 8 connection(s)
I20260811 19:00:01.988988 15595 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
I20260811 19:00:01.992043 15677 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:00:01.996135 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 15595
I20260811 19:00:01.996270 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/minicluster-data/master-0/wal/instance
I20260811 19:00:01.996565 13748 external_mini_cluster.cc:1428] Setting key 99cf24f4e60339648f5e1cb1f548465b
I20260811 19:00:01.998847 15677 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5c0646e359b477aaf79adc6b3f50482: Bootstrap starting.
I20260811 19:00:02.001627 15677 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c5c0646e359b477aaf79adc6b3f50482: Neither blocks nor log segments found. Creating new log.
I20260811 19:00:02.002915 15677 log.cc:824] T 00000000000000000000000000000000 P c5c0646e359b477aaf79adc6b3f50482: Log is configured to *not* fsync() on all Append() calls
I20260811 19:00:02.006258 15677 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5c0646e359b477aaf79adc6b3f50482: No bootstrap required, opened a new log
Aug 11 19:00:02 dist-test-slave-0kls krb5kdc[15582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474801, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
I20260811 19:00:02.012359 15677 raft_consensus.cc:348] T 00000000000000000000000000000000 P c5c0646e359b477aaf79adc6b3f50482 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5c0646e359b477aaf79adc6b3f50482" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 34491 } }
I20260811 19:00:02.012667 15677 raft_consensus.cc:374] T 00000000000000000000000000000000 P c5c0646e359b477aaf79adc6b3f50482 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:00:02.012732 15677 raft_consensus.cc:729] T 00000000000000000000000000000000 P c5c0646e359b477aaf79adc6b3f50482 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5c0646e359b477aaf79adc6b3f50482, State: Initialized, Role: FOLLOWER
I20260811 19:00:02.013268 15677 consensus_queue.cc:260] T 00000000000000000000000000000000 P c5c0646e359b477aaf79adc6b3f50482 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5c0646e359b477aaf79adc6b3f50482" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 34491 } }
I20260811 19:00:02.013441 15677 raft_consensus.cc:388] T 00000000000000000000000000000000 P c5c0646e359b477aaf79adc6b3f50482 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:00:02.013516 15677 raft_consensus.cc:482] T 00000000000000000000000000000000 P c5c0646e359b477aaf79adc6b3f50482 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:00:02.013638 15677 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c5c0646e359b477aaf79adc6b3f50482 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:00:02.015025 15677 raft_consensus.cc:504] T 00000000000000000000000000000000 P c5c0646e359b477aaf79adc6b3f50482 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5c0646e359b477aaf79adc6b3f50482" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 34491 } }
I20260811 19:00:02.015436 15677 leader_election.cc:302] T 00000000000000000000000000000000 P c5c0646e359b477aaf79adc6b3f50482 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c5c0646e359b477aaf79adc6b3f50482; no voters:
I20260811 19:00:02.015846 15677 leader_election.cc:288] T 00000000000000000000000000000000 P c5c0646e359b477aaf79adc6b3f50482 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:00:02.015972 15682 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c5c0646e359b477aaf79adc6b3f50482 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:00:02.016201 15680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:01.998263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58246 (local address 127.13.109.62:34491)
0811 19:00:01.998827 (+ 564us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:01.998846 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:01.998887 (+ 41us) server_negotiation.cc:407] Connection header received
0811 19:00:01.999747 (+ 860us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:01.999770 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:02.000184 (+ 414us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:02.000604 (+ 420us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:02.001635 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:02.003672 (+ 2037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:02.005261 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:02.006175 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:02.006867 (+ 692us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:02.009928 (+ 3061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:02.009933 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:02.009951 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:02.009985 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:02.012507 (+ 2522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:02.013080 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:02.013088 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:02.013094 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:02.013229 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:02.013574 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:02.013577 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:02.013579 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:02.014238 (+ 659us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:02.014449 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:02.014701 (+ 252us) server_negotiation.cc:299] Negotiation successful
0811 19:00:02.015068 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":163,"threads_started":1}
I20260811 19:00:02.016216 15682 raft_consensus.cc:686] T 00000000000000000000000000000000 P c5c0646e359b477aaf79adc6b3f50482 [term 1 LEADER]: Becoming Leader. State: Replica: c5c0646e359b477aaf79adc6b3f50482, State: Running, Role: LEADER
I20260811 19:00:02.016597 15682 consensus_queue.cc:237] T 00000000000000000000000000000000 P c5c0646e359b477aaf79adc6b3f50482 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5c0646e359b477aaf79adc6b3f50482" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 34491 } }
I20260811 19:00:02.017196 15677 sys_catalog.cc:564] T 00000000000000000000000000000000 P c5c0646e359b477aaf79adc6b3f50482 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:00:02.017985 15683 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5c0646e359b477aaf79adc6b3f50482 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c5c0646e359b477aaf79adc6b3f50482" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5c0646e359b477aaf79adc6b3f50482" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 34491 } } }
I20260811 19:00:02.018146 15683 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5c0646e359b477aaf79adc6b3f50482 [sys.catalog]: This master's current role is: LEADER
I20260811 19:00:02.018348 15684 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5c0646e359b477aaf79adc6b3f50482 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c5c0646e359b477aaf79adc6b3f50482. Latest consensus state: current_term: 1 leader_uuid: "c5c0646e359b477aaf79adc6b3f50482" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5c0646e359b477aaf79adc6b3f50482" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 34491 } } }
I20260811 19:00:02.018438 15684 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5c0646e359b477aaf79adc6b3f50482 [sys.catalog]: This master's current role is: LEADER
I20260811 19:00:02.018777 15689 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:00:02.022594 15689 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:00:02.028575 15689 catalog_manager.cc:1347] Generated new cluster ID: 666e67b4b0714d9f8ca13609dcdae58a
I20260811 19:00:02.028692 15689 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:00:02.039155 15689 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:00:02.040489 15689 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:00:02.048655 15689 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c5c0646e359b477aaf79adc6b3f50482: Generated new TSK 0
I20260811 19:00:02.049685 15689 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.109.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.109.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.109.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.1.keytab.
Entry for principal kudu/127.13.109.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.1.keytab.
I20260811 19:00:02.093367 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.1:0
--local_ip_for_outbound_sockets=127.13.109.1
--webserver_interface=127.13.109.1
--webserver_port=0
--tserver_master_addrs=127.13.109.62:34491
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.1.keytab
--principal=kudu/127.13.109.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:41169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:00:02.216655 15705 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:00:02.216972 15705 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:00:02.217022 15705 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:00:02.221540 15705 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:00:02.221616 15705 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:00:02.221661 15705 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 19:00:02.221676 15705 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 19:00:02.221771 15705 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.1
I20260811 19:00:02.227535 15705 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:41169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.1.keytab
--principal=kudu/127.13.109.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.109.1
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.109.62:34491
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15705
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.1
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:00:02.228852 15705 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:00:02.230013 15705 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:00:02.240625 15711 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:02.240789 15705 server_base.cc:1034] running on GCE node
W20260811 19:00:02.240535 15712 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:02.240540 15714 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:02.241482 15705 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:00:02.242336 15705 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:00:02.243510 15705 hybrid_clock.cc:648] HybridClock initialized: now 1786474802243491 us; error 33 us; skew 500 ppm
Aug 11 19:00:02 dist-test-slave-0kls krb5kdc[15582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474802, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:00:02.246436 15705 init.cc:377] Logged in from keytab as kudu/127.13.109.1@KRBTEST.COM (short username kudu)
I20260811 19:00:02.247393 15705 webserver.cc:460] Webserver started at http://127.13.109.1:36871/ using document root <none> and password file <none>
I20260811 19:00:02.247954 15705 fs_manager.cc:359] Metadata directory not provided
I20260811 19:00:02.248011 15705 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:00:02.248194 15705 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:00:02.250409 15705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/minicluster-data/ts-0/data/instance:
uuid: "9e08a2a95cd14e629ce94e2b5b75e6bd"
format_stamp: "Formatted at 2026-08-11 19:00:02 on dist-test-slave-0kls"
server_key: "55cd600ae5cf29a4abf6ae755230ff7c"
server_key_iv: "5830bd8abbc8a26fb20a6e0a9ca66c12"
server_key_version: "encryptionkey@0"
I20260811 19:00:02.251035 15705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/minicluster-data/ts-0/wal/instance:
uuid: "9e08a2a95cd14e629ce94e2b5b75e6bd"
format_stamp: "Formatted at 2026-08-11 19:00:02 on dist-test-slave-0kls"
server_key: "55cd600ae5cf29a4abf6ae755230ff7c"
server_key_iv: "5830bd8abbc8a26fb20a6e0a9ca66c12"
server_key_version: "encryptionkey@0"
I20260811 19:00:02.255069 15705 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260811 19:00:02.257655 15721 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:02.259027 15705 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 19:00:02.259198 15705 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/minicluster-data/ts-0/wal
uuid: "9e08a2a95cd14e629ce94e2b5b75e6bd"
format_stamp: "Formatted at 2026-08-11 19:00:02 on dist-test-slave-0kls"
server_key: "55cd600ae5cf29a4abf6ae755230ff7c"
server_key_iv: "5830bd8abbc8a26fb20a6e0a9ca66c12"
server_key_version: "encryptionkey@0"
I20260811 19:00:02.259326 15705 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:00:02.271036 15705 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:00:02.274441 15705 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:00:02.274689 15705 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:00:02.276433 15705 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:00:02.276489 15705 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:02.276530 15705 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:00:02.276580 15705 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:02.288452 15705 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.1:43961
I20260811 19:00:02.288545 15842 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.1:43961 every 8 connection(s)
Aug 11 19:00:02 dist-test-slave-0kls krb5kdc[15582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474802, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.1@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
I20260811 19:00:02.289498 15705 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
I20260811 19:00:02.291643 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 15705
I20260811 19:00:02.291749 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/minicluster-data/ts-0/wal/instance
I20260811 19:00:02.292100 13748 external_mini_cluster.cc:1428] Setting key 7fe74a20cfe5038e81dc845f781ad556
I20260811 19:00:02.293879 15680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:02.277178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.1:56709 (local address 127.13.109.62:34491)
0811 19:00:02.277366 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:02.277373 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:02.278903 (+ 1530us) server_negotiation.cc:407] Connection header received
0811 19:00:02.280121 (+ 1218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:02.280128 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:02.280205 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:02.280318 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:02.282540 (+ 2222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:02.283445 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:02.285639 (+ 2194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:02.285976 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:02.286122 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:02.289447 (+ 3325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:02.289451 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:02.289454 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:02.289497 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:02.291336 (+ 1839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:02.292005 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:02.292008 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:02.292011 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:02.292113 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:02.292497 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:02.292502 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:02.292507 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:02.292787 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:02.292894 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:02.293519 (+ 625us) server_negotiation.cc:299] Negotiation successful
0811 19:00:02.293681 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 19:00:02.294852 15731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:02.278210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:34491 (local address 127.13.109.1:56709)
0811 19:00:02.278714 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:02.278789 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:02.279893 (+ 1104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:02.280673 (+ 780us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:02.280684 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:02.281337 (+ 653us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:02.282277 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:02.282312 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:02.283634 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:02.283642 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:02.285452 (+ 1810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:02.285469 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:02.286396 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:02.286400 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:02.286680 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:02.287452 (+ 772us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:02.287483 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:02.289214 (+ 1731us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:00:02.291496 (+ 2282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:02.291501 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:02.291514 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:02.291893 (+ 379us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:02.292225 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:02.292228 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:02.292230 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:02.292391 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:02.292927 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:00:02.292935 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:00:02.293253 (+ 318us) client_negotiation.cc:769] Sending connection context
0811 19:00:02.293472 (+ 219us) client_negotiation.cc:240] Negotiation successful
0811 19:00:02.293957 (+ 485us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":136,"threads_started":1}
I20260811 19:00:02.296526 15843 heartbeater.cc:347] Connected to a master server at 127.13.109.62:34491
I20260811 19:00:02.296689 15843 heartbeater.cc:464] Registering TS with master...
I20260811 19:00:02.297350 15843 heartbeater.cc:511] Master 127.13.109.62:34491 requested a full tablet report, sending...
I20260811 19:00:02.299292 15627 ts_manager.cc:194] Registered new tserver with Master: 9e08a2a95cd14e629ce94e2b5b75e6bd (127.13.109.1:43961)
I20260811 19:00:02.300786 15627 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.109.1@KRBTEST.COM'} at 127.13.109.1:56709
WARNING: no policy specified for kudu/127.13.109.2@KRBTEST.COM; defaulting to no policy
I20260811 19:00:02.306823 15680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:02.299008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.1:59751 (local address 127.13.109.62:34491)
0811 19:00:02.299138 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:02.299142 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:02.299442 (+ 300us) server_negotiation.cc:407] Connection header received
0811 19:00:02.299592 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:02.299595 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:02.299642 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:02.299693 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:02.300387 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:02.300942 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:02.301973 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:02.302207 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:02.302315 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:02.303535 (+ 1220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:02.303539 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:02.303542 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:02.303566 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:02.305015 (+ 1449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:02.305499 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:02.305503 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:02.305504 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:02.305593 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:02.305915 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:02.305918 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:02.305920 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:02.306152 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:02.306260 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:02.306519 (+ 259us) server_negotiation.cc:299] Negotiation successful
0811 19:00:02.306650 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 19:00:02.306841 15851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:02.298929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:34491 (local address 127.13.109.1:59751)
0811 19:00:02.299322 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:02.299346 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:02.299458 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:02.299870 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:02.299873 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:02.300105 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:02.300234 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:02.300242 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:02.301081 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:02.301084 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:02.301850 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:02.301867 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:02.302333 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:02.302337 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:02.302490 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:02.303025 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:02.303037 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:02.303390 (+ 353us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:00:02.305134 (+ 1744us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:02.305139 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:02.305142 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:02.305398 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:02.305690 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:02.305693 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:02.305696 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:02.305831 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:02.306248 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:00:02.306252 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:00:02.306353 (+ 101us) client_negotiation.cc:769] Sending connection context
0811 19:00:02.306462 (+ 109us) client_negotiation.cc:240] Negotiation successful
0811 19:00:02.306615 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"mutex_wait_us":1,"thread_start_us":114,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.109.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.109.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.2.keytab.
Entry for principal kudu/127.13.109.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.2.keytab.
I20260811 19:00:02.325650 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.2:0
--local_ip_for_outbound_sockets=127.13.109.2
--webserver_interface=127.13.109.2
--webserver_port=0
--tserver_master_addrs=127.13.109.62:34491
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.2.keytab
--principal=kudu/127.13.109.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:41169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:00:02.451057 15853 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:00:02.451368 15853 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:00:02.451419 15853 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:00:02.455861 15853 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:00:02.455947 15853 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:00:02.455992 15853 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 19:00:02.456007 15853 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 19:00:02.456074 15853 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.2
I20260811 19:00:02.462036 15853 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:41169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.2.keytab
--principal=kudu/127.13.109.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.109.2
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.109.62:34491
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15853
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.2
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:00:02.463419 15853 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:00:02.464582 15853 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:00:02.475425 15860 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:02.475461 15863 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:02.475740 15853 server_base.cc:1034] running on GCE node
W20260811 19:00:02.475425 15861 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:02.476343 15853 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:00:02.477161 15853 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:00:02.478401 15853 hybrid_clock.cc:648] HybridClock initialized: now 1786474802478390 us; error 32 us; skew 500 ppm
Aug 11 19:00:02 dist-test-slave-0kls krb5kdc[15582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474802, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:00:02.481530 15853 init.cc:377] Logged in from keytab as kudu/127.13.109.2@KRBTEST.COM (short username kudu)
I20260811 19:00:02.482626 15853 webserver.cc:460] Webserver started at http://127.13.109.2:34739/ using document root <none> and password file <none>
I20260811 19:00:02.483229 15853 fs_manager.cc:359] Metadata directory not provided
I20260811 19:00:02.483340 15853 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:00:02.483577 15853 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:00:02.485721 15853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/minicluster-data/ts-1/data/instance:
uuid: "223b20a2034147dcba72be0e4a12c28a"
format_stamp: "Formatted at 2026-08-11 19:00:02 on dist-test-slave-0kls"
server_key: "2af274a46c40f2dd533e5f7cc5452334"
server_key_iv: "bfc9f7c1b988e235f3021dac4e0dc409"
server_key_version: "encryptionkey@0"
I20260811 19:00:02.486420 15853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/minicluster-data/ts-1/wal/instance:
uuid: "223b20a2034147dcba72be0e4a12c28a"
format_stamp: "Formatted at 2026-08-11 19:00:02 on dist-test-slave-0kls"
server_key: "2af274a46c40f2dd533e5f7cc5452334"
server_key_iv: "bfc9f7c1b988e235f3021dac4e0dc409"
server_key_version: "encryptionkey@0"
I20260811 19:00:02.490700 15853 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 19:00:02.493489 15872 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:02.495132 15853 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 19:00:02.495313 15853 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/minicluster-data/ts-1/wal
uuid: "223b20a2034147dcba72be0e4a12c28a"
format_stamp: "Formatted at 2026-08-11 19:00:02 on dist-test-slave-0kls"
server_key: "2af274a46c40f2dd533e5f7cc5452334"
server_key_iv: "bfc9f7c1b988e235f3021dac4e0dc409"
server_key_version: "encryptionkey@0"
I20260811 19:00:02.495460 15853 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:00:02.533417 15853 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:00:02.536888 15853 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:00:02.537184 15853 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:00:02.538857 15853 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:00:02.538909 15853 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:02.538975 15853 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:00:02.539005 15853 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 19:00:02 dist-test-slave-0kls krb5kdc[15582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474802, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.2@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
I20260811 19:00:02.551203 15853 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.2:40263
I20260811 19:00:02.551263 15996 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.2:40263 every 8 connection(s)
I20260811 19:00:02.552402 15853 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
I20260811 19:00:02.553994 15680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:02.539851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.2:45315 (local address 127.13.109.62:34491)
0811 19:00:02.539986 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:02.539989 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:02.541131 (+ 1142us) server_negotiation.cc:407] Connection header received
0811 19:00:02.542382 (+ 1251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:02.542387 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:02.542463 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:02.542572 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:02.543927 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:02.544531 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:02.545725 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:02.546122 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:02.546257 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:02.549572 (+ 3315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:02.549576 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:02.549578 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:02.549641 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:02.551550 (+ 1909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:02.552192 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:02.552194 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:02.552196 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:02.552273 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:02.552714 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:02.552716 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:02.552717 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:02.552984 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:02.553084 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:02.553684 (+ 600us) server_negotiation.cc:299] Negotiation successful
0811 19:00:02.553857 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 19:00:02.554785 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 15853
I20260811 19:00:02.554903 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/minicluster-data/ts-1/wal/instance
I20260811 19:00:02.554977 15881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:02.540430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:34491 (local address 127.13.109.2:45315)
0811 19:00:02.540956 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:02.541002 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:02.542093 (+ 1091us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:02.542791 (+ 698us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:02.542799 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:02.543208 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:02.543744 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:02.543798 (+ 54us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:02.544668 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:02.544672 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:02.545578 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:02.545588 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:02.546312 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:02.546317 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:02.546586 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:02.547306 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:02.547327 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:02.549335 (+ 2008us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:00:02.551709 (+ 2374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:02.551715 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:02.551729 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:02.552087 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:02.552372 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:02.552378 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:02.552382 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:02.552598 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:02.553142 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:00:02.553148 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:00:02.553438 (+ 290us) client_negotiation.cc:769] Sending connection context
0811 19:00:02.553601 (+ 163us) client_negotiation.cc:240] Negotiation successful
0811 19:00:02.553937 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":136,"threads_started":1}
I20260811 19:00:02.555245 13748 external_mini_cluster.cc:1428] Setting key 00d85e8e466ad8f779147556ef6f091e
I20260811 19:00:02.556567 15997 heartbeater.cc:347] Connected to a master server at 127.13.109.62:34491
I20260811 19:00:02.556728 15997 heartbeater.cc:464] Registering TS with master...
I20260811 19:00:02.557307 15997 heartbeater.cc:511] Master 127.13.109.62:34491 requested a full tablet report, sending...
I20260811 19:00:02.558444 15628 ts_manager.cc:194] Registered new tserver with Master: 223b20a2034147dcba72be0e4a12c28a (127.13.109.2:40263)
I20260811 19:00:02.559266 15628 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.109.2@KRBTEST.COM'} at 127.13.109.2:45315
WARNING: no policy specified for kudu/127.13.109.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.109.3@KRBTEST.COM" created.
I20260811 19:00:02.573777 15680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:02.559027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.2:39207 (local address 127.13.109.62:34491)
0811 19:00:02.559166 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:02.559171 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:02.566069 (+ 6898us) server_negotiation.cc:407] Connection header received
0811 19:00:02.566251 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:02.566253 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:02.566310 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:02.566387 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:02.567031 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:02.567558 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:02.568555 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:02.568860 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:02.568976 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:02.570210 (+ 1234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:02.570213 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:02.570216 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:02.570248 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:02.571976 (+ 1728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:02.572439 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:02.572443 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:02.572445 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:02.572528 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:02.572814 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:02.572818 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:02.572820 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:02.573073 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:02.573191 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:02.573470 (+ 279us) server_negotiation.cc:299] Negotiation successful
0811 19:00:02.573590 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 19:00:02.573784 16005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:02.558933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:34491 (local address 127.13.109.2:39207)
0811 19:00:02.559255 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:02.565921 (+ 6666us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:02.566136 (+ 215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:02.566460 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:02.566465 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:02.566717 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:02.566888 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:02.566898 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:02.567708 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:02.567715 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:02.568443 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:02.568454 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:02.568989 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:02.568993 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:02.569095 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:02.569610 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:02.569623 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:02.570050 (+ 427us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:00:02.572108 (+ 2058us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:02.572113 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:02.572115 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:02.572335 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:02.572624 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:02.572627 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:02.572629 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:02.572721 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:02.573183 (+ 462us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:00:02.573186 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:00:02.573294 (+ 108us) client_negotiation.cc:769] Sending connection context
0811 19:00:02.573401 (+ 107us) client_negotiation.cc:240] Negotiation successful
0811 19:00:02.573575 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":92,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.109.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.3.keytab.
Entry for principal kudu/127.13.109.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.3.keytab.
I20260811 19:00:02.588637 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.3:0
--local_ip_for_outbound_sockets=127.13.109.3
--webserver_interface=127.13.109.3
--webserver_port=0
--tserver_master_addrs=127.13.109.62:34491
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.3.keytab
--principal=kudu/127.13.109.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:41169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:00:02.712592 16007 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:00:02.712906 16007 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:00:02.712957 16007 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:00:02.717285 16007 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:00:02.717362 16007 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:00:02.717461 16007 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 19:00:02.717499 16007 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 19:00:02.717571 16007 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.3
I20260811 19:00:02.723577 16007 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:41169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.3.keytab
--principal=kudu/127.13.109.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.109.3
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.109.62:34491
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16007
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.3
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:00:02.724905 16007 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:00:02.726199 16007 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:00:02.736855 16014 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:02.736855 16016 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:02.737103 16007 server_base.cc:1034] running on GCE node
W20260811 19:00:02.736855 16012 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:02.737702 16007 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:00:02.738417 16007 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:00:02.739598 16007 hybrid_clock.cc:648] HybridClock initialized: now 1786474802739577 us; error 31 us; skew 500 ppm
Aug 11 19:00:02 dist-test-slave-0kls krb5kdc[15582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474802, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:00:02.742759 16007 init.cc:377] Logged in from keytab as kudu/127.13.109.3@KRBTEST.COM (short username kudu)
I20260811 19:00:02.743739 16007 webserver.cc:460] Webserver started at http://127.13.109.3:43209/ using document root <none> and password file <none>
I20260811 19:00:02.744395 16007 fs_manager.cc:359] Metadata directory not provided
I20260811 19:00:02.744464 16007 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:00:02.744701 16007 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:00:02.746902 16007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/minicluster-data/ts-2/data/instance:
uuid: "ff643294d6ac4617b8d13a10a12df80f"
format_stamp: "Formatted at 2026-08-11 19:00:02 on dist-test-slave-0kls"
server_key: "b70d114fbc139e335c3f6ec88b1f3eb8"
server_key_iv: "11e5ebbcb3a536566c9e31c4850366f6"
server_key_version: "encryptionkey@0"
I20260811 19:00:02.747491 16007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/minicluster-data/ts-2/wal/instance:
uuid: "ff643294d6ac4617b8d13a10a12df80f"
format_stamp: "Formatted at 2026-08-11 19:00:02 on dist-test-slave-0kls"
server_key: "b70d114fbc139e335c3f6ec88b1f3eb8"
server_key_iv: "11e5ebbcb3a536566c9e31c4850366f6"
server_key_version: "encryptionkey@0"
I20260811 19:00:02.751446 16007 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 19:00:02.754125 16023 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:02.755520 16007 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 19:00:02.755702 16007 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/minicluster-data/ts-2/wal
uuid: "ff643294d6ac4617b8d13a10a12df80f"
format_stamp: "Formatted at 2026-08-11 19:00:02 on dist-test-slave-0kls"
server_key: "b70d114fbc139e335c3f6ec88b1f3eb8"
server_key_iv: "11e5ebbcb3a536566c9e31c4850366f6"
server_key_version: "encryptionkey@0"
I20260811 19:00:02.755849 16007 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:00:02.776192 16007 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:00:02.779426 16007 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:00:02.779683 16007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:00:02.781350 16007 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:00:02.781437 16007 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:02.781539 16007 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:00:02.781590 16007 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 19:00:02 dist-test-slave-0kls krb5kdc[15582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474802, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.3@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
I20260811 19:00:02.793741 16007 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.3:43919
I20260811 19:00:02.793871 16147 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.3:43919 every 8 connection(s)
I20260811 19:00:02.795085 16007 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
I20260811 19:00:02.796896 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 16007
I20260811 19:00:02.797050 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/minicluster-data/ts-2/wal/instance
I20260811 19:00:02.797353 13748 external_mini_cluster.cc:1428] Setting key 9d273b659639b419761544e2a1351492
I20260811 19:00:02.797947 15680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:02.782307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.3:34443 (local address 127.13.109.62:34491)
0811 19:00:02.782475 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:02.782480 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:02.783540 (+ 1060us) server_negotiation.cc:407] Connection header received
0811 19:00:02.785108 (+ 1568us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:02.785113 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:02.785209 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:02.785311 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:02.786801 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:02.787368 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:02.789359 (+ 1991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:02.789763 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:02.789910 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:02.793430 (+ 3520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:02.793433 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:02.793436 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:02.793497 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:02.795385 (+ 1888us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:02.796065 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:02.796068 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:02.796070 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:02.796156 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:02.796576 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:02.796580 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:02.796581 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:02.796849 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:02.796969 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:02.797577 (+ 608us) server_negotiation.cc:299] Negotiation successful
0811 19:00:02.797726 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 19:00:02.798851 16032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:02.782751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:34491 (local address 127.13.109.3:34443)
0811 19:00:02.783339 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:02.783406 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:02.784801 (+ 1395us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:02.785502 (+ 701us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:02.785510 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:02.785998 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:02.786626 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:02.786643 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:02.787543 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:02.787550 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:02.789201 (+ 1651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:02.789215 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:02.790014 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:02.790022 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:02.790295 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:02.791034 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:02.791059 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:02.793223 (+ 2164us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:00:02.795531 (+ 2308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:02.795538 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:02.795555 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:02.795917 (+ 362us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:02.796274 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:02.796277 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:02.796279 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:02.796434 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:02.796968 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:00:02.796975 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:00:02.797306 (+ 331us) client_negotiation.cc:769] Sending connection context
0811 19:00:02.797518 (+ 212us) client_negotiation.cc:240] Negotiation successful
0811 19:00:02.797878 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":108,"threads_started":1}
I20260811 19:00:02.800235 16148 heartbeater.cc:347] Connected to a master server at 127.13.109.62:34491
I20260811 19:00:02.800431 16148 heartbeater.cc:464] Registering TS with master...
I20260811 19:00:02.801044 16148 heartbeater.cc:511] Master 127.13.109.62:34491 requested a full tablet report, sending...
I20260811 19:00:02.802460 15627 ts_manager.cc:194] Registered new tserver with Master: ff643294d6ac4617b8d13a10a12df80f (127.13.109.3:43919)
I20260811 19:00:02.803298 15627 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.109.3@KRBTEST.COM'} at 127.13.109.3:34443
I20260811 19:00:02.810245 15680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:02.803093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.3:46181 (local address 127.13.109.62:34491)
0811 19:00:02.803235 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:02.803239 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:02.803451 (+ 212us) server_negotiation.cc:407] Connection header received
0811 19:00:02.803547 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:02.803550 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:02.803621 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:02.803706 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:02.804266 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:02.805006 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:02.805897 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:02.806111 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:02.806194 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:02.807274 (+ 1080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:02.807278 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:02.807280 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:02.807311 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:02.808525 (+ 1214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:02.808972 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:02.808976 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:02.808980 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:02.809062 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:02.809346 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:02.809350 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:02.809351 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:02.809532 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:02.809650 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:02.809961 (+ 311us) server_negotiation.cc:299] Negotiation successful
0811 19:00:02.810076 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 19:00:02.810290 16154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:02.802934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:34491 (local address 127.13.109.3:46181)
0811 19:00:02.803306 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:02.803325 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:02.803446 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:02.803745 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:02.803748 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:02.804004 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:02.804138 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:02.804145 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:02.805164 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:02.805166 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:02.805727 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:02.805735 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:02.806214 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:02.806216 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:02.806285 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:02.806772 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:02.806783 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:02.807114 (+ 331us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:00:02.808652 (+ 1538us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:02.808655 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:02.808657 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:02.808863 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:02.809161 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:02.809164 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:02.809166 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:02.809253 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:02.809635 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:00:02.809638 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:00:02.809742 (+ 104us) client_negotiation.cc:769] Sending connection context
0811 19:00:02.809901 (+ 159us) client_negotiation.cc:240] Negotiation successful
0811 19:00:02.810063 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"mutex_wait_us":2,"thread_start_us":128,"threads_started":1}
I20260811 19:00:02.811895 13748 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 19:00:02 dist-test-slave-0kls krb5kdc[15582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474802, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 19:00:02 dist-test-slave-0kls krb5kdc[15582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474802, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
I20260811 19:00:02.840706 15680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:02.831345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58252 (local address 127.13.109.62:34491)
0811 19:00:02.831574 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:02.831581 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:02.831604 (+ 23us) server_negotiation.cc:407] Connection header received
0811 19:00:02.831678 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:02.831683 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:02.831749 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:02.831859 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:02.832828 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:02.833371 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:02.834351 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:02.834570 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:02.834666 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:02.837257 (+ 2591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:02.837260 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:02.837263 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:02.837295 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:02.838882 (+ 1587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:02.839431 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:02.839434 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:02.839435 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:02.839524 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:02.839804 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:02.839806 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:02.839807 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:02.840021 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:02.840127 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:02.840362 (+ 235us) server_negotiation.cc:299] Negotiation successful
0811 19:00:02.840474 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260811 19:00:02.844321 15628 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58252:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 19:00:02.847450 15628 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 19:00:02.863662 16165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:02.857322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.1:43961 (local address 127.0.0.1:44398)
0811 19:00:02.858246 (+ 924us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:02.858303 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:02.858433 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:02.859240 (+ 807us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:02.859246 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:02.859269 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:00:02.859498 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:02.859505 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:02.860820 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:02.860824 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:02.861988 (+ 1164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:02.861998 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:02.863033 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:02.863037 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:02.863159 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:02.863203 (+ 44us) client_negotiation.cc:769] Sending connection context
0811 19:00:02.863342 (+ 139us) client_negotiation.cc:240] Negotiation successful
0811 19:00:02.863469 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":751,"mutex_wait_us":3,"thread_start_us":84,"threads_started":1}
I20260811 19:00:02.863674 16166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:02.857453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44398 (local address 127.13.109.1:43961)
0811 19:00:02.858747 (+ 1294us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:02.858754 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:02.858789 (+ 35us) server_negotiation.cc:407] Connection header received
0811 19:00:02.858866 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:02.858871 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:02.859073 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:02.859245 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:00:02.859645 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:02.860678 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:02.862118 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:02.862862 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:02.862982 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:02.863135 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:02.863395 (+ 260us) server_negotiation.cc:299] Negotiation successful
0811 19:00:02.863511 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1181,"thread_start_us":109,"threads_started":1}
I20260811 19:00:02.864297 16167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:02.858140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.3:43919 (local address 127.0.0.1:36028)
0811 19:00:02.858676 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:02.858689 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:02.858770 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:02.859503 (+ 733us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:02.859506 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:02.859525 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:00:02.859628 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:02.859633 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:02.861008 (+ 1375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:02.861010 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:02.862574 (+ 1564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:02.862582 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:02.863879 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:02.863885 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:02.863970 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:02.863995 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 19:00:02.864085 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 19:00:02.864165 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"mutex_wait_us":164,"thread_start_us":105,"threads_started":1}
I20260811 19:00:02.864789 16170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:02.858594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36028 (local address 127.13.109.3:43919)
0811 19:00:02.859056 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:02.859069 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:02.859109 (+ 40us) server_negotiation.cc:407] Connection header received
0811 19:00:02.859180 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:02.859185 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:02.859351 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:02.859504 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:00:02.859759 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:02.860884 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:02.862706 (+ 1822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:02.863463 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:02.863566 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:02.863809 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:02.864526 (+ 717us) server_negotiation.cc:299] Negotiation successful
0811 19:00:02.864632 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":123,"threads_started":1}
I20260811 19:00:02.864980 16168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:02.857723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53002 (local address 127.13.109.2:40263)
0811 19:00:02.858124 (+ 401us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:02.858137 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:02.858485 (+ 348us) server_negotiation.cc:407] Connection header received
0811 19:00:02.858749 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:02.858755 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:02.858981 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:02.859133 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:00:02.859622 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:02.860857 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:02.862723 (+ 1866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:02.863562 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:02.863664 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:02.864228 (+ 564us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:02.864737 (+ 509us) server_negotiation.cc:299] Negotiation successful
0811 19:00:02.864842 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":267,"thread_start_us":140,"threads_started":1}
I20260811 19:00:02.865392 16169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:02.857521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.2:40263 (local address 127.0.0.1:53002)
0811 19:00:02.858377 (+ 856us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:02.858394 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:02.858580 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:02.859328 (+ 748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:02.859331 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:02.859348 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:00:02.859477 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:02.859484 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:02.861007 (+ 1523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:02.861010 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:02.862570 (+ 1560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:02.862582 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:02.864293 (+ 1711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:02.864296 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:02.864566 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:02.864590 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 19:00:02.865196 (+ 606us) client_negotiation.cc:240] Negotiation successful
0811 19:00:02.865275 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":788,"mutex_wait_us":154,"thread_start_us":116,"threads_started":1}
I20260811 19:00:02.867266 15775 tablet_service.cc:1467] Processing CreateTablet for tablet 7c44d6635d3644bda8d1785cb3a9e148 (DEFAULT_TABLE table=test-table [id=10f3294d7f704a9eb3ffe11e7ffc9685]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:00:02.867631 16076 tablet_service.cc:1467] Processing CreateTablet for tablet 7c44d6635d3644bda8d1785cb3a9e148 (DEFAULT_TABLE table=test-table [id=10f3294d7f704a9eb3ffe11e7ffc9685]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:00:02.868435 15775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c44d6635d3644bda8d1785cb3a9e148. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:00:02.868104 15926 tablet_service.cc:1467] Processing CreateTablet for tablet 7c44d6635d3644bda8d1785cb3a9e148 (DEFAULT_TABLE table=test-table [id=10f3294d7f704a9eb3ffe11e7ffc9685]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:00:02.868763 16076 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c44d6635d3644bda8d1785cb3a9e148. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:00:02.869093 15926 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c44d6635d3644bda8d1785cb3a9e148. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:00:02.875643 16171 tablet_bootstrap.cc:492] T 7c44d6635d3644bda8d1785cb3a9e148 P 9e08a2a95cd14e629ce94e2b5b75e6bd: Bootstrap starting.
I20260811 19:00:02.876663 16172 tablet_bootstrap.cc:492] T 7c44d6635d3644bda8d1785cb3a9e148 P 223b20a2034147dcba72be0e4a12c28a: Bootstrap starting.
I20260811 19:00:02.878093 16173 tablet_bootstrap.cc:492] T 7c44d6635d3644bda8d1785cb3a9e148 P ff643294d6ac4617b8d13a10a12df80f: Bootstrap starting.
I20260811 19:00:02.878443 16171 tablet_bootstrap.cc:654] T 7c44d6635d3644bda8d1785cb3a9e148 P 9e08a2a95cd14e629ce94e2b5b75e6bd: Neither blocks nor log segments found. Creating new log.
I20260811 19:00:02.878952 16172 tablet_bootstrap.cc:654] T 7c44d6635d3644bda8d1785cb3a9e148 P 223b20a2034147dcba72be0e4a12c28a: Neither blocks nor log segments found. Creating new log.
I20260811 19:00:02.879406 16171 log.cc:824] T 7c44d6635d3644bda8d1785cb3a9e148 P 9e08a2a95cd14e629ce94e2b5b75e6bd: Log is configured to *not* fsync() on all Append() calls
I20260811 19:00:02.879838 16172 log.cc:824] T 7c44d6635d3644bda8d1785cb3a9e148 P 223b20a2034147dcba72be0e4a12c28a: Log is configured to *not* fsync() on all Append() calls
I20260811 19:00:02.880234 16173 tablet_bootstrap.cc:654] T 7c44d6635d3644bda8d1785cb3a9e148 P ff643294d6ac4617b8d13a10a12df80f: Neither blocks nor log segments found. Creating new log.
I20260811 19:00:02.881063 16173 log.cc:824] T 7c44d6635d3644bda8d1785cb3a9e148 P ff643294d6ac4617b8d13a10a12df80f: Log is configured to *not* fsync() on all Append() calls
I20260811 19:00:02.881903 16171 tablet_bootstrap.cc:492] T 7c44d6635d3644bda8d1785cb3a9e148 P 9e08a2a95cd14e629ce94e2b5b75e6bd: No bootstrap required, opened a new log
I20260811 19:00:02.882166 16171 ts_tablet_manager.cc:1397] T 7c44d6635d3644bda8d1785cb3a9e148 P 9e08a2a95cd14e629ce94e2b5b75e6bd: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260811 19:00:02.882189 16172 tablet_bootstrap.cc:492] T 7c44d6635d3644bda8d1785cb3a9e148 P 223b20a2034147dcba72be0e4a12c28a: No bootstrap required, opened a new log
I20260811 19:00:02.882388 16172 ts_tablet_manager.cc:1397] T 7c44d6635d3644bda8d1785cb3a9e148 P 223b20a2034147dcba72be0e4a12c28a: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 19:00:02.883358 16173 tablet_bootstrap.cc:492] T 7c44d6635d3644bda8d1785cb3a9e148 P ff643294d6ac4617b8d13a10a12df80f: No bootstrap required, opened a new log
I20260811 19:00:02.883563 16173 ts_tablet_manager.cc:1397] T 7c44d6635d3644bda8d1785cb3a9e148 P ff643294d6ac4617b8d13a10a12df80f: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 19:00:02.886405 16171 raft_consensus.cc:348] T 7c44d6635d3644bda8d1785cb3a9e148 P 9e08a2a95cd14e629ce94e2b5b75e6bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e08a2a95cd14e629ce94e2b5b75e6bd" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 43961 } } peers { permanent_uuid: "ff643294d6ac4617b8d13a10a12df80f" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43919 } } peers { permanent_uuid: "223b20a2034147dcba72be0e4a12c28a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 40263 } }
I20260811 19:00:02.886734 16171 raft_consensus.cc:374] T 7c44d6635d3644bda8d1785cb3a9e148 P 9e08a2a95cd14e629ce94e2b5b75e6bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:00:02.886525 16172 raft_consensus.cc:348] T 7c44d6635d3644bda8d1785cb3a9e148 P 223b20a2034147dcba72be0e4a12c28a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e08a2a95cd14e629ce94e2b5b75e6bd" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 43961 } } peers { permanent_uuid: "ff643294d6ac4617b8d13a10a12df80f" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43919 } } peers { permanent_uuid: "223b20a2034147dcba72be0e4a12c28a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 40263 } }
I20260811 19:00:02.886795 16171 raft_consensus.cc:729] T 7c44d6635d3644bda8d1785cb3a9e148 P 9e08a2a95cd14e629ce94e2b5b75e6bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e08a2a95cd14e629ce94e2b5b75e6bd, State: Initialized, Role: FOLLOWER
I20260811 19:00:02.886814 16172 raft_consensus.cc:374] T 7c44d6635d3644bda8d1785cb3a9e148 P 223b20a2034147dcba72be0e4a12c28a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:00:02.886870 16172 raft_consensus.cc:729] T 7c44d6635d3644bda8d1785cb3a9e148 P 223b20a2034147dcba72be0e4a12c28a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 223b20a2034147dcba72be0e4a12c28a, State: Initialized, Role: FOLLOWER
I20260811 19:00:02.887301 16171 consensus_queue.cc:260] T 7c44d6635d3644bda8d1785cb3a9e148 P 9e08a2a95cd14e629ce94e2b5b75e6bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e08a2a95cd14e629ce94e2b5b75e6bd" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 43961 } } peers { permanent_uuid: "ff643294d6ac4617b8d13a10a12df80f" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43919 } } peers { permanent_uuid: "223b20a2034147dcba72be0e4a12c28a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 40263 } }
I20260811 19:00:02.887369 16172 consensus_queue.cc:260] T 7c44d6635d3644bda8d1785cb3a9e148 P 223b20a2034147dcba72be0e4a12c28a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e08a2a95cd14e629ce94e2b5b75e6bd" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 43961 } } peers { permanent_uuid: "ff643294d6ac4617b8d13a10a12df80f" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43919 } } peers { permanent_uuid: "223b20a2034147dcba72be0e4a12c28a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 40263 } }
I20260811 19:00:02.888109 15843 heartbeater.cc:503] Master 127.13.109.62:34491 was elected leader, sending a full tablet report...
I20260811 19:00:02.888432 16171 ts_tablet_manager.cc:1428] T 7c44d6635d3644bda8d1785cb3a9e148 P 9e08a2a95cd14e629ce94e2b5b75e6bd: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 19:00:02.888453 16172 ts_tablet_manager.cc:1428] T 7c44d6635d3644bda8d1785cb3a9e148 P 223b20a2034147dcba72be0e4a12c28a: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 19:00:02.888840 15997 heartbeater.cc:503] Master 127.13.109.62:34491 was elected leader, sending a full tablet report...
I20260811 19:00:02.887861 16173 raft_consensus.cc:348] T 7c44d6635d3644bda8d1785cb3a9e148 P ff643294d6ac4617b8d13a10a12df80f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e08a2a95cd14e629ce94e2b5b75e6bd" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 43961 } } peers { permanent_uuid: "ff643294d6ac4617b8d13a10a12df80f" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43919 } } peers { permanent_uuid: "223b20a2034147dcba72be0e4a12c28a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 40263 } }
I20260811 19:00:02.889499 16173 raft_consensus.cc:374] T 7c44d6635d3644bda8d1785cb3a9e148 P ff643294d6ac4617b8d13a10a12df80f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:00:02.889582 16173 raft_consensus.cc:729] T 7c44d6635d3644bda8d1785cb3a9e148 P ff643294d6ac4617b8d13a10a12df80f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff643294d6ac4617b8d13a10a12df80f, State: Initialized, Role: FOLLOWER
I20260811 19:00:02.890148 16173 consensus_queue.cc:260] T 7c44d6635d3644bda8d1785cb3a9e148 P ff643294d6ac4617b8d13a10a12df80f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e08a2a95cd14e629ce94e2b5b75e6bd" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 43961 } } peers { permanent_uuid: "ff643294d6ac4617b8d13a10a12df80f" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43919 } } peers { permanent_uuid: "223b20a2034147dcba72be0e4a12c28a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 40263 } }
I20260811 19:00:02.890993 16148 heartbeater.cc:503] Master 127.13.109.62:34491 was elected leader, sending a full tablet report...
I20260811 19:00:02.891402 16173 ts_tablet_manager.cc:1428] T 7c44d6635d3644bda8d1785cb3a9e148 P ff643294d6ac4617b8d13a10a12df80f: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260811 19:00:03.010031 16177 raft_consensus.cc:482] T 7c44d6635d3644bda8d1785cb3a9e148 P 9e08a2a95cd14e629ce94e2b5b75e6bd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:00:03.010260 16177 raft_consensus.cc:504] T 7c44d6635d3644bda8d1785cb3a9e148 P 9e08a2a95cd14e629ce94e2b5b75e6bd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e08a2a95cd14e629ce94e2b5b75e6bd" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 43961 } } peers { permanent_uuid: "ff643294d6ac4617b8d13a10a12df80f" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43919 } } peers { permanent_uuid: "223b20a2034147dcba72be0e4a12c28a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 40263 } }
I20260811 19:00:03.011484 16177 leader_election.cc:288] T 7c44d6635d3644bda8d1785cb3a9e148 P 9e08a2a95cd14e629ce94e2b5b75e6bd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ff643294d6ac4617b8d13a10a12df80f (127.13.109.3:43919), 223b20a2034147dcba72be0e4a12c28a (127.13.109.2:40263)
I20260811 19:00:03.015758 16168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:03.011916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.1:58173 (local address 127.13.109.2:40263)
0811 19:00:03.012090 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:03.012095 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:03.012328 (+ 233us) server_negotiation.cc:407] Connection header received
0811 19:00:03.012502 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:03.012511 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:03.012555 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:03.012658 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:00:03.012960 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:03.013495 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:03.014740 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:03.015147 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:03.015284 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:03.015345 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:03.015541 (+ 196us) server_negotiation.cc:299] Negotiation successful
0811 19:00:03.015625 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 19:00:03.015774 16170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:03.011990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.1:42057 (local address 127.13.109.3:43919)
0811 19:00:03.012151 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:03.012156 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:03.012378 (+ 222us) server_negotiation.cc:407] Connection header received
0811 19:00:03.012546 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:03.012549 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:03.012600 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:03.012691 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:00:03.012994 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:03.013526 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:03.014721 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:03.015150 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:03.015293 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:03.015353 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:03.015569 (+ 216us) server_negotiation.cc:299] Negotiation successful
0811 19:00:03.015643 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 19:00:03.015795 16180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:03.011760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.2:40263 (local address 127.13.109.1:58173)
0811 19:00:03.012220 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:03.012234 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:03.012386 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:03.012673 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:03.012675 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:03.012691 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:00:03.012842 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:03.012848 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:03.013623 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:03.013627 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:03.014613 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:03.014623 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:03.015280 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:03.015284 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:03.015401 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:03.015435 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 19:00:03.015523 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 19:00:03.015603 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":388,"mutex_wait_us":49,"spinlock_wait_cycles":28288,"thread_start_us":134,"threads_started":1}
I20260811 19:00:03.015801 16181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:03.011760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.3:43919 (local address 127.13.109.1:42057)
0811 19:00:03.012245 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:03.012263 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:03.012414 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:03.012704 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:03.012707 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:03.012748 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:00:03.012872 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:03.012878 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:03.013658 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:03.013663 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:03.014579 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:03.014589 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:03.015285 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:03.015288 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:03.015439 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:03.015457 (+ 18us) client_negotiation.cc:769] Sending connection context
0811 19:00:03.015548 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 19:00:03.015621 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":429,"mutex_wait_us":149,"spinlock_wait_cycles":896,"thread_start_us":117,"threads_started":1}
I20260811 19:00:03.016579 15947 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c44d6635d3644bda8d1785cb3a9e148" candidate_uuid: "9e08a2a95cd14e629ce94e2b5b75e6bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "223b20a2034147dcba72be0e4a12c28a" is_pre_election: true
I20260811 19:00:03.016752 16097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c44d6635d3644bda8d1785cb3a9e148" candidate_uuid: "9e08a2a95cd14e629ce94e2b5b75e6bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff643294d6ac4617b8d13a10a12df80f" is_pre_election: true
I20260811 19:00:03.017004 15947 raft_consensus.cc:2454] T 7c44d6635d3644bda8d1785cb3a9e148 P 223b20a2034147dcba72be0e4a12c28a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e08a2a95cd14e629ce94e2b5b75e6bd in term 0.
I20260811 19:00:03.017068 16097 raft_consensus.cc:2454] T 7c44d6635d3644bda8d1785cb3a9e148 P ff643294d6ac4617b8d13a10a12df80f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e08a2a95cd14e629ce94e2b5b75e6bd in term 0.
I20260811 19:00:03.017602 15724 leader_election.cc:302] T 7c44d6635d3644bda8d1785cb3a9e148 P 9e08a2a95cd14e629ce94e2b5b75e6bd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 223b20a2034147dcba72be0e4a12c28a, 9e08a2a95cd14e629ce94e2b5b75e6bd; no voters:
I20260811 19:00:03.017959 16177 raft_consensus.cc:2781] T 7c44d6635d3644bda8d1785cb3a9e148 P 9e08a2a95cd14e629ce94e2b5b75e6bd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 19:00:03.018052 16177 raft_consensus.cc:482] T 7c44d6635d3644bda8d1785cb3a9e148 P 9e08a2a95cd14e629ce94e2b5b75e6bd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 19:00:03.018111 16177 raft_consensus.cc:3037] T 7c44d6635d3644bda8d1785cb3a9e148 P 9e08a2a95cd14e629ce94e2b5b75e6bd [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:00:03.019323 16177 raft_consensus.cc:504] T 7c44d6635d3644bda8d1785cb3a9e148 P 9e08a2a95cd14e629ce94e2b5b75e6bd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e08a2a95cd14e629ce94e2b5b75e6bd" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 43961 } } peers { permanent_uuid: "ff643294d6ac4617b8d13a10a12df80f" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43919 } } peers { permanent_uuid: "223b20a2034147dcba72be0e4a12c28a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 40263 } }
I20260811 19:00:03.019744 16177 leader_election.cc:288] T 7c44d6635d3644bda8d1785cb3a9e148 P 9e08a2a95cd14e629ce94e2b5b75e6bd [CANDIDATE]: Term 1 election: Requested vote from peers ff643294d6ac4617b8d13a10a12df80f (127.13.109.3:43919), 223b20a2034147dcba72be0e4a12c28a (127.13.109.2:40263)
I20260811 19:00:03.020129 15947 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c44d6635d3644bda8d1785cb3a9e148" candidate_uuid: "9e08a2a95cd14e629ce94e2b5b75e6bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "223b20a2034147dcba72be0e4a12c28a"
I20260811 19:00:03.020135 16097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c44d6635d3644bda8d1785cb3a9e148" candidate_uuid: "9e08a2a95cd14e629ce94e2b5b75e6bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff643294d6ac4617b8d13a10a12df80f"
I20260811 19:00:03.020295 15947 raft_consensus.cc:3037] T 7c44d6635d3644bda8d1785cb3a9e148 P 223b20a2034147dcba72be0e4a12c28a [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:00:03.020303 16097 raft_consensus.cc:3037] T 7c44d6635d3644bda8d1785cb3a9e148 P ff643294d6ac4617b8d13a10a12df80f [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:00:03.021463 15947 raft_consensus.cc:2454] T 7c44d6635d3644bda8d1785cb3a9e148 P 223b20a2034147dcba72be0e4a12c28a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e08a2a95cd14e629ce94e2b5b75e6bd in term 1.
I20260811 19:00:03.021726 16097 raft_consensus.cc:2454] T 7c44d6635d3644bda8d1785cb3a9e148 P ff643294d6ac4617b8d13a10a12df80f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e08a2a95cd14e629ce94e2b5b75e6bd in term 1.
I20260811 19:00:03.021893 15724 leader_election.cc:302] T 7c44d6635d3644bda8d1785cb3a9e148 P 9e08a2a95cd14e629ce94e2b5b75e6bd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 223b20a2034147dcba72be0e4a12c28a, 9e08a2a95cd14e629ce94e2b5b75e6bd; no voters:
I20260811 19:00:03.022131 16177 raft_consensus.cc:2781] T 7c44d6635d3644bda8d1785cb3a9e148 P 9e08a2a95cd14e629ce94e2b5b75e6bd [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:00:03.022378 16177 raft_consensus.cc:686] T 7c44d6635d3644bda8d1785cb3a9e148 P 9e08a2a95cd14e629ce94e2b5b75e6bd [term 1 LEADER]: Becoming Leader. State: Replica: 9e08a2a95cd14e629ce94e2b5b75e6bd, State: Running, Role: LEADER
I20260811 19:00:03.022782 16177 consensus_queue.cc:237] T 7c44d6635d3644bda8d1785cb3a9e148 P 9e08a2a95cd14e629ce94e2b5b75e6bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e08a2a95cd14e629ce94e2b5b75e6bd" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 43961 } } peers { permanent_uuid: "ff643294d6ac4617b8d13a10a12df80f" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43919 } } peers { permanent_uuid: "223b20a2034147dcba72be0e4a12c28a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 40263 } }
I20260811 19:00:03.026371 15627 catalog_manager.cc:5495] T 7c44d6635d3644bda8d1785cb3a9e148 P 9e08a2a95cd14e629ce94e2b5b75e6bd reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e08a2a95cd14e629ce94e2b5b75e6bd (127.13.109.1). New cstate: current_term: 1 leader_uuid: "9e08a2a95cd14e629ce94e2b5b75e6bd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e08a2a95cd14e629ce94e2b5b75e6bd" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 43961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff643294d6ac4617b8d13a10a12df80f" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "223b20a2034147dcba72be0e4a12c28a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 40263 } health_report { overall_health: UNKNOWN } } }
W20260811 19:00:03.041199 15844 tablet.cc:2369] T 7c44d6635d3644bda8d1785cb3a9e148 P 9e08a2a95cd14e629ce94e2b5b75e6bd: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 19:00:03.046468 16149 tablet.cc:2369] T 7c44d6635d3644bda8d1785cb3a9e148 P ff643294d6ac4617b8d13a10a12df80f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 19:00:03.053776 15998 tablet.cc:2369] T 7c44d6635d3644bda8d1785cb3a9e148 P 223b20a2034147dcba72be0e4a12c28a: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 11 19:00:03 dist-test-slave-0kls krb5kdc[15582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474801, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.62@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
I20260811 19:00:03.101972 16189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:03.091837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:34491 (local address 127.0.0.1:58264)
0811 19:00:03.092237 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:03.092251 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:03.092410 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:03.092707 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:03.092711 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:03.093194 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:03.093299 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:03.093307 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:03.094282 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:03.094287 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:03.094907 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:03.094916 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:03.095423 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:03.095426 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:03.095506 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:03.096126 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:03.096140 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:03.097937 (+ 1797us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:00:03.099919 (+ 1982us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:03.099940 (+ 21us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:03.099947 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:03.100363 (+ 416us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:03.100719 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:03.100722 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:03.100723 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:03.100816 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:03.101339 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:00:03.101345 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:00:03.101479 (+ 134us) client_negotiation.cc:769] Sending connection context
0811 19:00:03.101594 (+ 115us) client_negotiation.cc:240] Negotiation successful
0811 19:00:03.101736 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":141,"threads_started":1}
I20260811 19:00:03.102078 15680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:03.091936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58264 (local address 127.13.109.62:34491)
0811 19:00:03.092132 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:03.092139 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:03.092340 (+ 201us) server_negotiation.cc:407] Connection header received
0811 19:00:03.092530 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:03.092533 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:03.092587 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:03.092684 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:03.093447 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:03.094152 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:03.095052 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:03.095286 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:03.095412 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:03.098129 (+ 2717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:03.098133 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:03.098135 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:03.098165 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:03.099721 (+ 1556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:03.100454 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:03.100458 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:03.100459 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:03.100547 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:03.100942 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:03.100945 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:03.100946 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:03.101161 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:03.101334 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:03.101712 (+ 378us) server_negotiation.cc:299] Negotiation successful
0811 19:00:03.101906 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 19:00:03.106998 15627 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.13.109.62@KRBTEST.COM'} at 127.0.0.1:58264:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 19:00:03.107822 15627 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 19:00:03.113337 15775 tablet_service.cc:1467] Processing CreateTablet for tablet 8f7f4d320c2a4fa39d653bbd00ab47a4 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d08efed6bc034c939c9b395c9e21c26c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 19:00:03.113335 16076 tablet_service.cc:1467] Processing CreateTablet for tablet 8f7f4d320c2a4fa39d653bbd00ab47a4 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d08efed6bc034c939c9b395c9e21c26c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 19:00:03.113306 15926 tablet_service.cc:1467] Processing CreateTablet for tablet 8f7f4d320c2a4fa39d653bbd00ab47a4 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d08efed6bc034c939c9b395c9e21c26c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 19:00:03.114136 15775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f7f4d320c2a4fa39d653bbd00ab47a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:00:03.114137 15926 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f7f4d320c2a4fa39d653bbd00ab47a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:00:03.114138 16076 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f7f4d320c2a4fa39d653bbd00ab47a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:00:03.116662 16171 tablet_bootstrap.cc:492] T 8f7f4d320c2a4fa39d653bbd00ab47a4 P 9e08a2a95cd14e629ce94e2b5b75e6bd: Bootstrap starting.
I20260811 19:00:03.117424 16173 tablet_bootstrap.cc:492] T 8f7f4d320c2a4fa39d653bbd00ab47a4 P ff643294d6ac4617b8d13a10a12df80f: Bootstrap starting.
I20260811 19:00:03.118476 16171 tablet_bootstrap.cc:654] T 8f7f4d320c2a4fa39d653bbd00ab47a4 P 9e08a2a95cd14e629ce94e2b5b75e6bd: Neither blocks nor log segments found. Creating new log.
I20260811 19:00:03.118628 16172 tablet_bootstrap.cc:492] T 8f7f4d320c2a4fa39d653bbd00ab47a4 P 223b20a2034147dcba72be0e4a12c28a: Bootstrap starting.
I20260811 19:00:03.118638 16173 tablet_bootstrap.cc:654] T 8f7f4d320c2a4fa39d653bbd00ab47a4 P ff643294d6ac4617b8d13a10a12df80f: Neither blocks nor log segments found. Creating new log.
I20260811 19:00:03.119915 16172 tablet_bootstrap.cc:654] T 8f7f4d320c2a4fa39d653bbd00ab47a4 P 223b20a2034147dcba72be0e4a12c28a: Neither blocks nor log segments found. Creating new log.
I20260811 19:00:03.120038 16171 tablet_bootstrap.cc:492] T 8f7f4d320c2a4fa39d653bbd00ab47a4 P 9e08a2a95cd14e629ce94e2b5b75e6bd: No bootstrap required, opened a new log
I20260811 19:00:03.120121 16171 ts_tablet_manager.cc:1397] T 8f7f4d320c2a4fa39d653bbd00ab47a4 P 9e08a2a95cd14e629ce94e2b5b75e6bd: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.002s
I20260811 19:00:03.120168 16173 tablet_bootstrap.cc:492] T 8f7f4d320c2a4fa39d653bbd00ab47a4 P ff643294d6ac4617b8d13a10a12df80f: No bootstrap required, opened a new log
I20260811 19:00:03.120268 16173 ts_tablet_manager.cc:1397] T 8f7f4d320c2a4fa39d653bbd00ab47a4 P ff643294d6ac4617b8d13a10a12df80f: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 19:00:03.120682 16171 raft_consensus.cc:348] T 8f7f4d320c2a4fa39d653bbd00ab47a4 P 9e08a2a95cd14e629ce94e2b5b75e6bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e08a2a95cd14e629ce94e2b5b75e6bd" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 43961 } } peers { permanent_uuid: "223b20a2034147dcba72be0e4a12c28a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 40263 } } peers { permanent_uuid: "ff643294d6ac4617b8d13a10a12df80f" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43919 } }
I20260811 19:00:03.120816 16171 raft_consensus.cc:374] T 8f7f4d320c2a4fa39d653bbd00ab47a4 P 9e08a2a95cd14e629ce94e2b5b75e6bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:00:03.120853 16171 raft_consensus.cc:729] T 8f7f4d320c2a4fa39d653bbd00ab47a4 P 9e08a2a95cd14e629ce94e2b5b75e6bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e08a2a95cd14e629ce94e2b5b75e6bd, State: Initialized, Role: FOLLOWER
I20260811 19:00:03.120893 16173 raft_consensus.cc:348] T 8f7f4d320c2a4fa39d653bbd00ab47a4 P ff643294d6ac4617b8d13a10a12df80f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e08a2a95cd14e629ce94e2b5b75e6bd" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 43961 } } peers { permanent_uuid: "223b20a2034147dcba72be0e4a12c28a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 40263 } } peers { permanent_uuid: "ff643294d6ac4617b8d13a10a12df80f" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43919 } }
I20260811 19:00:03.121059 16173 raft_consensus.cc:374] T 8f7f4d320c2a4fa39d653bbd00ab47a4 P ff643294d6ac4617b8d13a10a12df80f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:00:03.121026 16171 consensus_queue.cc:260] T 8f7f4d320c2a4fa39d653bbd00ab47a4 P 9e08a2a95cd14e629ce94e2b5b75e6bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e08a2a95cd14e629ce94e2b5b75e6bd" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 43961 } } peers { permanent_uuid: "223b20a2034147dcba72be0e4a12c28a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 40263 } } peers { permanent_uuid: "ff643294d6ac4617b8d13a10a12df80f" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43919 } }
I20260811 19:00:03.121119 16173 raft_consensus.cc:729] T 8f7f4d320c2a4fa39d653bbd00ab47a4 P ff643294d6ac4617b8d13a10a12df80f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff643294d6ac4617b8d13a10a12df80f, State: Initialized, Role: FOLLOWER
I20260811 19:00:03.121271 16173 consensus_queue.cc:260] T 8f7f4d320c2a4fa39d653bbd00ab47a4 P ff643294d6ac4617b8d13a10a12df80f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e08a2a95cd14e629ce94e2b5b75e6bd" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 43961 } } peers { permanent_uuid: "223b20a2034147dcba72be0e4a12c28a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 40263 } } peers { permanent_uuid: "ff643294d6ac4617b8d13a10a12df80f" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43919 } }
I20260811 19:00:03.121358 16177 tablet_replica.cc:439] T 8f7f4d320c2a4fa39d653bbd00ab47a4 P 9e08a2a95cd14e629ce94e2b5b75e6bd: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e08a2a95cd14e629ce94e2b5b75e6bd" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 43961 } } peers { permanent_uuid: "223b20a2034147dcba72be0e4a12c28a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 40263 } } peers { permanent_uuid: "ff643294d6ac4617b8d13a10a12df80f" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43919 } } }
I20260811 19:00:03.121460 16172 tablet_bootstrap.cc:492] T 8f7f4d320c2a4fa39d653bbd00ab47a4 P 223b20a2034147dcba72be0e4a12c28a: No bootstrap required, opened a new log
I20260811 19:00:03.121496 16177 tablet_replica.cc:442] T 8f7f4d320c2a4fa39d653bbd00ab47a4 P 9e08a2a95cd14e629ce94e2b5b75e6bd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 19:00:03.121557 16172 ts_tablet_manager.cc:1397] T 8f7f4d320c2a4fa39d653bbd00ab47a4 P 223b20a2034147dcba72be0e4a12c28a: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 19:00:03.121619 16173 ts_tablet_manager.cc:1428] T 8f7f4d320c2a4fa39d653bbd00ab47a4 P ff643294d6ac4617b8d13a10a12df80f: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260811 19:00:03.121740 16171 ts_tablet_manager.cc:1428] T 8f7f4d320c2a4fa39d653bbd00ab47a4 P 9e08a2a95cd14e629ce94e2b5b75e6bd: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260811 19:00:03.121984 16179 tablet_replica.cc:439] T 8f7f4d320c2a4fa39d653bbd00ab47a4 P ff643294d6ac4617b8d13a10a12df80f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e08a2a95cd14e629ce94e2b5b75e6bd" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 43961 } } peers { permanent_uuid: "223b20a2034147dcba72be0e4a12c28a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 40263 } } peers { permanent_uuid: "ff643294d6ac4617b8d13a10a12df80f" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43919 } } }
I20260811 19:00:03.122189 16179 tablet_replica.cc:442] T 8f7f4d320c2a4fa39d653bbd00ab47a4 P ff643294d6ac4617b8d13a10a12df80f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 19:00:03.122581 16172 raft_consensus.cc:348] T 8f7f4d320c2a4fa39d653bbd00ab47a4 P 223b20a2034147dcba72be0e4a12c28a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e08a2a95cd14e629ce94e2b5b75e6bd" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 43961 } } peers { permanent_uuid: "223b20a2034147dcba72be0e4a12c28a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 40263 } } peers { permanent_uuid: "ff643294d6ac4617b8d13a10a12df80f" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43919 } }
I20260811 19:00:03.122727 16172 raft_consensus.cc:374] T 8f7f4d320c2a4fa39d653bbd00ab47a4 P 223b20a2034147dcba72be0e4a12c28a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:00:03.122778 16172 raft_consensus.cc:729] T 8f7f4d320c2a4fa39d653bbd00ab47a4 P 223b20a2034147dcba72be0e4a12c28a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 223b20a2034147dcba72be0e4a12c28a, State: Initialized, Role: FOLLOWER
I20260811 19:00:03.122943 16172 consensus_queue.cc:260] T 8f7f4d320c2a4fa39d653bbd00ab47a4 P 223b20a2034147dcba72be0e4a12c28a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e08a2a95cd14e629ce94e2b5b75e6bd" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 43961 } } peers { permanent_uuid: "223b20a2034147dcba72be0e4a12c28a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 40263 } } peers { permanent_uuid: "ff643294d6ac4617b8d13a10a12df80f" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43919 } }
I20260811 19:00:03.123291 16172 ts_tablet_manager.cc:1428] T 8f7f4d320c2a4fa39d653bbd00ab47a4 P 223b20a2034147dcba72be0e4a12c28a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 19:00:03.123569 16178 tablet_replica.cc:439] T 8f7f4d320c2a4fa39d653bbd00ab47a4 P 223b20a2034147dcba72be0e4a12c28a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e08a2a95cd14e629ce94e2b5b75e6bd" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 43961 } } peers { permanent_uuid: "223b20a2034147dcba72be0e4a12c28a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 40263 } } peers { permanent_uuid: "ff643294d6ac4617b8d13a10a12df80f" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43919 } } }
I20260811 19:00:03.123773 16178 tablet_replica.cc:442] T 8f7f4d320c2a4fa39d653bbd00ab47a4 P 223b20a2034147dcba72be0e4a12c28a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 19:00:03.237633 16179 raft_consensus.cc:482] T 8f7f4d320c2a4fa39d653bbd00ab47a4 P ff643294d6ac4617b8d13a10a12df80f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:00:03.237833 16179 raft_consensus.cc:504] T 8f7f4d320c2a4fa39d653bbd00ab47a4 P ff643294d6ac4617b8d13a10a12df80f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e08a2a95cd14e629ce94e2b5b75e6bd" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 43961 } } peers { permanent_uuid: "223b20a2034147dcba72be0e4a12c28a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 40263 } } peers { permanent_uuid: "ff643294d6ac4617b8d13a10a12df80f" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43919 } }
I20260811 19:00:03.239111 16179 leader_election.cc:288] T 8f7f4d320c2a4fa39d653bbd00ab47a4 P ff643294d6ac4617b8d13a10a12df80f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e08a2a95cd14e629ce94e2b5b75e6bd (127.13.109.1:43961), 223b20a2034147dcba72be0e4a12c28a (127.13.109.2:40263)
I20260811 19:00:03.243180 16032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:03.239383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.2:40263 (local address 127.13.109.3:59349)
0811 19:00:03.239507 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:03.239525 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:03.239649 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:03.240003 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:03.240007 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:03.240030 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:00:03.240160 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:03.240166 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:03.241137 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:03.241143 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:03.242027 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:03.242037 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:03.242709 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:03.242713 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:03.242832 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:03.242852 (+ 20us) client_negotiation.cc:769] Sending connection context
0811 19:00:03.242943 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 19:00:03.243024 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260811 19:00:03.243180 16168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:03.239569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.3:59349 (local address 127.13.109.2:40263)
0811 19:00:03.239754 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:03.239757 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:03.239771 (+ 14us) server_negotiation.cc:407] Connection header received
0811 19:00:03.239825 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:03.239828 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:03.239868 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:03.239941 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:00:03.240304 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:03.241016 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:03.242167 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:03.242572 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:03.242709 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:03.242769 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:03.242965 (+ 196us) server_negotiation.cc:299] Negotiation successful
0811 19:00:03.243043 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 19:00:03.243577 16166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:03.239466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.3:56533 (local address 127.13.109.1:43961)
0811 19:00:03.239628 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:03.239635 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:03.240364 (+ 729us) server_negotiation.cc:407] Connection header received
0811 19:00:03.240428 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:03.240431 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:03.240474 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:03.240553 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:00:03.240866 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:03.241356 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:03.242507 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:03.242938 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:03.243084 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:03.243161 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:03.243348 (+ 187us) server_negotiation.cc:299] Negotiation successful
0811 19:00:03.243453 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 19:00:03.243814 15947 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f7f4d320c2a4fa39d653bbd00ab47a4" candidate_uuid: "ff643294d6ac4617b8d13a10a12df80f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "223b20a2034147dcba72be0e4a12c28a" is_pre_election: true
I20260811 19:00:03.243975 16193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:03.239388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.1:43961 (local address 127.13.109.3:56533)
0811 19:00:03.240197 (+ 809us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:03.240210 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:03.240354 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:03.240595 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:03.240598 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:03.240617 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:00:03.240742 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:03.240748 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:03.241496 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:03.241502 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:03.242373 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:03.242383 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:03.243084 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:03.243088 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:03.243193 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:03.243214 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 19:00:03.243736 (+ 522us) client_negotiation.cc:240] Negotiation successful
0811 19:00:03.243816 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":746,"mutex_wait_us":352,"thread_start_us":163,"threads_started":1}
I20260811 19:00:03.244001 15947 raft_consensus.cc:2454] T 8f7f4d320c2a4fa39d653bbd00ab47a4 P 223b20a2034147dcba72be0e4a12c28a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff643294d6ac4617b8d13a10a12df80f in term 0.
I20260811 19:00:03.244451 16026 leader_election.cc:302] T 8f7f4d320c2a4fa39d653bbd00ab47a4 P ff643294d6ac4617b8d13a10a12df80f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 223b20a2034147dcba72be0e4a12c28a, ff643294d6ac4617b8d13a10a12df80f; no voters:
I20260811 19:00:03.244712 15795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f7f4d320c2a4fa39d653bbd00ab47a4" candidate_uuid: "ff643294d6ac4617b8d13a10a12df80f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e08a2a95cd14e629ce94e2b5b75e6bd" is_pre_election: true
I20260811 19:00:03.244863 16179 raft_consensus.cc:2781] T 8f7f4d320c2a4fa39d653bbd00ab47a4 P ff643294d6ac4617b8d13a10a12df80f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 19:00:03.244937 16179 raft_consensus.cc:482] T 8f7f4d320c2a4fa39d653bbd00ab47a4 P ff643294d6ac4617b8d13a10a12df80f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 19:00:03.244989 16179 raft_consensus.cc:3037] T 8f7f4d320c2a4fa39d653bbd00ab47a4 P ff643294d6ac4617b8d13a10a12df80f [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:00:03.244966 15795 raft_consensus.cc:2454] T 8f7f4d320c2a4fa39d653bbd00ab47a4 P 9e08a2a95cd14e629ce94e2b5b75e6bd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff643294d6ac4617b8d13a10a12df80f in term 0.
I20260811 19:00:03.246158 16179 raft_consensus.cc:504] T 8f7f4d320c2a4fa39d653bbd00ab47a4 P ff643294d6ac4617b8d13a10a12df80f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e08a2a95cd14e629ce94e2b5b75e6bd" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 43961 } } peers { permanent_uuid: "223b20a2034147dcba72be0e4a12c28a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 40263 } } peers { permanent_uuid: "ff643294d6ac4617b8d13a10a12df80f" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43919 } }
I20260811 19:00:03.246558 16179 leader_election.cc:288] T 8f7f4d320c2a4fa39d653bbd00ab47a4 P ff643294d6ac4617b8d13a10a12df80f [CANDIDATE]: Term 1 election: Requested vote from peers 9e08a2a95cd14e629ce94e2b5b75e6bd (127.13.109.1:43961), 223b20a2034147dcba72be0e4a12c28a (127.13.109.2:40263)
I20260811 19:00:03.246942 15947 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f7f4d320c2a4fa39d653bbd00ab47a4" candidate_uuid: "ff643294d6ac4617b8d13a10a12df80f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "223b20a2034147dcba72be0e4a12c28a"
I20260811 19:00:03.246949 15795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f7f4d320c2a4fa39d653bbd00ab47a4" candidate_uuid: "ff643294d6ac4617b8d13a10a12df80f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e08a2a95cd14e629ce94e2b5b75e6bd"
I20260811 19:00:03.247053 15795 raft_consensus.cc:3037] T 8f7f4d320c2a4fa39d653bbd00ab47a4 P 9e08a2a95cd14e629ce94e2b5b75e6bd [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:00:03.247085 15947 raft_consensus.cc:3037] T 8f7f4d320c2a4fa39d653bbd00ab47a4 P 223b20a2034147dcba72be0e4a12c28a [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:00:03.248049 15795 raft_consensus.cc:2454] T 8f7f4d320c2a4fa39d653bbd00ab47a4 P 9e08a2a95cd14e629ce94e2b5b75e6bd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff643294d6ac4617b8d13a10a12df80f in term 1.
I20260811 19:00:03.248054 15947 raft_consensus.cc:2454] T 8f7f4d320c2a4fa39d653bbd00ab47a4 P 223b20a2034147dcba72be0e4a12c28a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff643294d6ac4617b8d13a10a12df80f in term 1.
I20260811 19:00:03.248374 16027 leader_election.cc:302] T 8f7f4d320c2a4fa39d653bbd00ab47a4 P ff643294d6ac4617b8d13a10a12df80f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e08a2a95cd14e629ce94e2b5b75e6bd, ff643294d6ac4617b8d13a10a12df80f; no voters:
I20260811 19:00:03.248551 16179 raft_consensus.cc:2781] T 8f7f4d320c2a4fa39d653bbd00ab47a4 P ff643294d6ac4617b8d13a10a12df80f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:00:03.248780 16179 raft_consensus.cc:686] T 8f7f4d320c2a4fa39d653bbd00ab47a4 P ff643294d6ac4617b8d13a10a12df80f [term 1 LEADER]: Becoming Leader. State: Replica: ff643294d6ac4617b8d13a10a12df80f, State: Running, Role: LEADER
I20260811 19:00:03.249137 16179 consensus_queue.cc:237] T 8f7f4d320c2a4fa39d653bbd00ab47a4 P ff643294d6ac4617b8d13a10a12df80f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e08a2a95cd14e629ce94e2b5b75e6bd" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 43961 } } peers { permanent_uuid: "223b20a2034147dcba72be0e4a12c28a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 40263 } } peers { permanent_uuid: "ff643294d6ac4617b8d13a10a12df80f" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43919 } }
I20260811 19:00:03.251474 16194 tablet_replica.cc:439] T 8f7f4d320c2a4fa39d653bbd00ab47a4 P ff643294d6ac4617b8d13a10a12df80f: TxnStatusTablet state changed. Reason: New leader ff643294d6ac4617b8d13a10a12df80f. Latest consensus state: current_term: 1 leader_uuid: "ff643294d6ac4617b8d13a10a12df80f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e08a2a95cd14e629ce94e2b5b75e6bd" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 43961 } } peers { permanent_uuid: "223b20a2034147dcba72be0e4a12c28a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 40263 } } peers { permanent_uuid: "ff643294d6ac4617b8d13a10a12df80f" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43919 } } }
I20260811 19:00:03.251636 16194 tablet_replica.cc:442] T 8f7f4d320c2a4fa39d653bbd00ab47a4 P ff643294d6ac4617b8d13a10a12df80f: This TxnStatusTablet replica's current role is: LEADER
I20260811 19:00:03.251951 16196 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 19:00:03.252174 16196 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 19:00:03.252745 15627 catalog_manager.cc:5495] T 8f7f4d320c2a4fa39d653bbd00ab47a4 P ff643294d6ac4617b8d13a10a12df80f reported cstate change: term changed from 0 to 1, leader changed from <none> to ff643294d6ac4617b8d13a10a12df80f (127.13.109.3). New cstate: current_term: 1 leader_uuid: "ff643294d6ac4617b8d13a10a12df80f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e08a2a95cd14e629ce94e2b5b75e6bd" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 43961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "223b20a2034147dcba72be0e4a12c28a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 40263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff643294d6ac4617b8d13a10a12df80f" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43919 } health_report { overall_health: HEALTHY } } }
I20260811 19:00:03.286816 16170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:03.282484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36040 (local address 127.13.109.3:43919)
0811 19:00:03.282632 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:03.282636 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:03.282651 (+ 15us) server_negotiation.cc:407] Connection header received
0811 19:00:03.282899 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:03.282902 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:03.282964 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:03.283058 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:00:03.283347 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:03.283860 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:03.285117 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:03.285337 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:03.285419 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:03.285874 (+ 455us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:00:03.286372 (+ 498us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:00:03.286474 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:03.286604 (+ 130us) server_negotiation.cc:299] Negotiation successful
0811 19:00:03.286667 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 19:00:03.286839 16189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:03.282320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.3:43919 (local address 127.0.0.1:36040)
0811 19:00:03.282567 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:03.282585 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:03.282721 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:03.283098 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:03.283101 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:03.283110 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 19:00:03.283234 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:03.283240 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:03.283970 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:03.283974 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:03.284977 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:03.284989 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:03.285455 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:03.285459 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:03.285553 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:03.285626 (+ 73us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 19:00:03.286478 (+ 852us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 19:00:03.286498 (+ 20us) client_negotiation.cc:769] Sending connection context
0811 19:00:03.286577 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 19:00:03.286686 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107}
I20260811 19:00:03.293248 15947 raft_consensus.cc:1261] T 8f7f4d320c2a4fa39d653bbd00ab47a4 P 223b20a2034147dcba72be0e4a12c28a [term 1 FOLLOWER]: Refusing update from remote peer ff643294d6ac4617b8d13a10a12df80f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:00:03.293248 15795 raft_consensus.cc:1261] T 8f7f4d320c2a4fa39d653bbd00ab47a4 P 9e08a2a95cd14e629ce94e2b5b75e6bd [term 1 FOLLOWER]: Refusing update from remote peer ff643294d6ac4617b8d13a10a12df80f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:00:03.294003 16179 consensus_queue.cc:1035] T 8f7f4d320c2a4fa39d653bbd00ab47a4 P ff643294d6ac4617b8d13a10a12df80f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e08a2a95cd14e629ce94e2b5b75e6bd" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 43961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:00:03.294303 16194 consensus_queue.cc:1035] T 8f7f4d320c2a4fa39d653bbd00ab47a4 P ff643294d6ac4617b8d13a10a12df80f [LEADER]: Connected to new peer: Peer: permanent_uuid: "223b20a2034147dcba72be0e4a12c28a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 40263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:00:03.298043 16177 tablet_replica.cc:439] T 8f7f4d320c2a4fa39d653bbd00ab47a4 P 9e08a2a95cd14e629ce94e2b5b75e6bd: TxnStatusTablet state changed. Reason: New leader ff643294d6ac4617b8d13a10a12df80f. Latest consensus state: current_term: 1 leader_uuid: "ff643294d6ac4617b8d13a10a12df80f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e08a2a95cd14e629ce94e2b5b75e6bd" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 43961 } } peers { permanent_uuid: "223b20a2034147dcba72be0e4a12c28a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 40263 } } peers { permanent_uuid: "ff643294d6ac4617b8d13a10a12df80f" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43919 } } }
I20260811 19:00:03.298178 16177 tablet_replica.cc:442] T 8f7f4d320c2a4fa39d653bbd00ab47a4 P 9e08a2a95cd14e629ce94e2b5b75e6bd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 19:00:03.304924 16179 tablet_replica.cc:439] T 8f7f4d320c2a4fa39d653bbd00ab47a4 P ff643294d6ac4617b8d13a10a12df80f: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ff643294d6ac4617b8d13a10a12df80f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e08a2a95cd14e629ce94e2b5b75e6bd" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 43961 } } peers { permanent_uuid: "223b20a2034147dcba72be0e4a12c28a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 40263 } } peers { permanent_uuid: "ff643294d6ac4617b8d13a10a12df80f" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43919 } } }
I20260811 19:00:03.305076 16179 tablet_replica.cc:442] T 8f7f4d320c2a4fa39d653bbd00ab47a4 P ff643294d6ac4617b8d13a10a12df80f: This TxnStatusTablet replica's current role is: LEADER
I20260811 19:00:03.305905 16200 mvcc.cc:205] Tried to move back new op lower bound from 7317400794281676800 to 7317400794111303680. Current Snapshot: MvccSnapshot[applied={T|T < 7317400794281676800}]
I20260811 19:00:03.305992 16177 tablet_replica.cc:439] T 8f7f4d320c2a4fa39d653bbd00ab47a4 P 9e08a2a95cd14e629ce94e2b5b75e6bd: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ff643294d6ac4617b8d13a10a12df80f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e08a2a95cd14e629ce94e2b5b75e6bd" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 43961 } } peers { permanent_uuid: "223b20a2034147dcba72be0e4a12c28a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 40263 } } peers { permanent_uuid: "ff643294d6ac4617b8d13a10a12df80f" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43919 } } }
I20260811 19:00:03.306118 16177 tablet_replica.cc:442] T 8f7f4d320c2a4fa39d653bbd00ab47a4 P 9e08a2a95cd14e629ce94e2b5b75e6bd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 19:00:03.306115 16178 tablet_replica.cc:439] T 8f7f4d320c2a4fa39d653bbd00ab47a4 P 223b20a2034147dcba72be0e4a12c28a: TxnStatusTablet state changed. Reason: New leader ff643294d6ac4617b8d13a10a12df80f. Latest consensus state: current_term: 1 leader_uuid: "ff643294d6ac4617b8d13a10a12df80f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e08a2a95cd14e629ce94e2b5b75e6bd" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 43961 } } peers { permanent_uuid: "223b20a2034147dcba72be0e4a12c28a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 40263 } } peers { permanent_uuid: "ff643294d6ac4617b8d13a10a12df80f" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43919 } } }
I20260811 19:00:03.306226 16178 tablet_replica.cc:442] T 8f7f4d320c2a4fa39d653bbd00ab47a4 P 223b20a2034147dcba72be0e4a12c28a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 19:00:03.307058 16201 mvcc.cc:205] Tried to move back new op lower bound from 7317400794281676800 to 7317400794111303680. Current Snapshot: MvccSnapshot[applied={T|T < 7317400794281676800}]
I20260811 19:00:03.308060 16179 tablet_replica.cc:439] T 8f7f4d320c2a4fa39d653bbd00ab47a4 P ff643294d6ac4617b8d13a10a12df80f: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ff643294d6ac4617b8d13a10a12df80f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e08a2a95cd14e629ce94e2b5b75e6bd" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 43961 } } peers { permanent_uuid: "223b20a2034147dcba72be0e4a12c28a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 40263 } } peers { permanent_uuid: "ff643294d6ac4617b8d13a10a12df80f" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43919 } } }
I20260811 19:00:03.308183 16179 tablet_replica.cc:442] T 8f7f4d320c2a4fa39d653bbd00ab47a4 P ff643294d6ac4617b8d13a10a12df80f: This TxnStatusTablet replica's current role is: LEADER
I20260811 19:00:03.309659 16203 mvcc.cc:205] Tried to move back new op lower bound from 7317400794281676800 to 7317400794111303680. Current Snapshot: MvccSnapshot[applied={T|T < 7317400794281676800}]
I20260811 19:00:03.310750 16178 tablet_replica.cc:439] T 8f7f4d320c2a4fa39d653bbd00ab47a4 P 223b20a2034147dcba72be0e4a12c28a: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ff643294d6ac4617b8d13a10a12df80f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e08a2a95cd14e629ce94e2b5b75e6bd" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 43961 } } peers { permanent_uuid: "223b20a2034147dcba72be0e4a12c28a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 40263 } } peers { permanent_uuid: "ff643294d6ac4617b8d13a10a12df80f" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43919 } } }
I20260811 19:00:03.310868 16178 tablet_replica.cc:442] T 8f7f4d320c2a4fa39d653bbd00ab47a4 P 223b20a2034147dcba72be0e4a12c28a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 19:00:03.318432 16166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:03.313888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44406 (local address 127.13.109.1:43961)
0811 19:00:03.314005 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:03.314008 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:03.314022 (+ 14us) server_negotiation.cc:407] Connection header received
0811 19:00:03.314196 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:03.314199 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:03.314264 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:03.314375 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:00:03.314608 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:03.315386 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:03.316311 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:03.316672 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:03.316779 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:03.317298 (+ 519us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:00:03.317887 (+ 589us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:00:03.318019 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:03.318149 (+ 130us) server_negotiation.cc:299] Negotiation successful
0811 19:00:03.318216 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260811 19:00:03.331806 16170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:03.328148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.1:50797 (local address 127.13.109.3:43919)
0811 19:00:03.328312 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:03.328334 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:03.328436 (+ 102us) server_negotiation.cc:407] Connection header received
0811 19:00:03.328651 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:03.328655 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:03.328713 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:03.328792 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:00:03.329088 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:03.329724 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:03.330644 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:03.330880 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:03.330987 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:03.331334 (+ 347us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:00:03.331417 (+ 83us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:00:03.331494 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:03.331603 (+ 109us) server_negotiation.cc:299] Negotiation successful
0811 19:00:03.331654 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 19:00:03.331815 16212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:03.327997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.3:43919 (local address 127.13.109.1:50797)
0811 19:00:03.328335 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:03.328351 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:03.328514 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:03.328817 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:03.328821 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:03.328828 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 19:00:03.328970 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:03.328976 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:03.329864 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:03.329868 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:03.330507 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:03.330517 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:03.331004 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:03.331008 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:03.331115 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:03.331162 (+ 47us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 19:00:03.331500 (+ 338us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 19:00:03.331509 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 19:00:03.331590 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 19:00:03.331677 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":103,"threads_started":1}
I20260811 19:00:03.340444 15946 raft_consensus.cc:1261] T 7c44d6635d3644bda8d1785cb3a9e148 P 223b20a2034147dcba72be0e4a12c28a [term 1 FOLLOWER]: Refusing update from remote peer 9e08a2a95cd14e629ce94e2b5b75e6bd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:00:03.340684 16097 raft_consensus.cc:1261] T 7c44d6635d3644bda8d1785cb3a9e148 P ff643294d6ac4617b8d13a10a12df80f [term 1 FOLLOWER]: Refusing update from remote peer 9e08a2a95cd14e629ce94e2b5b75e6bd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:00:03.341317 16177 consensus_queue.cc:1035] T 7c44d6635d3644bda8d1785cb3a9e148 P 9e08a2a95cd14e629ce94e2b5b75e6bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff643294d6ac4617b8d13a10a12df80f" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:00:03.341508 16182 consensus_queue.cc:1035] T 7c44d6635d3644bda8d1785cb3a9e148 P 9e08a2a95cd14e629ce94e2b5b75e6bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "223b20a2034147dcba72be0e4a12c28a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 40263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:00:03.368132 16166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:03.363679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.3:53607 (local address 127.13.109.1:43961)
0811 19:00:03.363823 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:03.363828 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:03.364087 (+ 259us) server_negotiation.cc:407] Connection header received
0811 19:00:03.364270 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:03.364273 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:03.364336 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:03.364415 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:00:03.364809 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:03.365571 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:03.366911 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:03.367129 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:03.367213 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:03.367565 (+ 352us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:00:03.367659 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:00:03.367731 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:03.367930 (+ 199us) server_negotiation.cc:299] Negotiation successful
0811 19:00:03.368001 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 19:00:03.368144 16218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:03.363609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.1:43961 (local address 127.13.109.3:53607)
0811 19:00:03.363988 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:03.364009 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:03.364138 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:03.364545 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:03.364549 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:03.364557 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 19:00:03.364666 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:03.364673 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:03.365721 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:03.365724 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:03.366777 (+ 1053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:03.366802 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:03.367252 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:03.367256 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:03.367348 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:03.367404 (+ 56us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 19:00:03.367754 (+ 350us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 19:00:03.367761 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 19:00:03.367862 (+ 101us) client_negotiation.cc:240] Negotiation successful
0811 19:00:03.367970 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":140,"threads_started":1}
I20260811 19:00:03.396078 15628 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58252:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:00:03.396294 15628 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58252:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:00:03.399413 15628 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c5c0646e359b477aaf79adc6b3f50482: Sending DeleteTablet for 3 replicas of tablet 7c44d6635d3644bda8d1785cb3a9e148
I20260811 19:00:03.400454 16076 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c44d6635d3644bda8d1785cb3a9e148 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:00:03 UTC) from {username='kudu', principal='kudu/127.13.109.62@KRBTEST.COM'} at 127.0.0.1:36028
I20260811 19:00:03.400543 15926 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c44d6635d3644bda8d1785cb3a9e148 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:00:03 UTC) from {username='kudu', principal='kudu/127.13.109.62@KRBTEST.COM'} at 127.0.0.1:53002
I20260811 19:00:03.401500 15775 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c44d6635d3644bda8d1785cb3a9e148 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:00:03 UTC) from {username='kudu', principal='kudu/127.13.109.62@KRBTEST.COM'} at 127.0.0.1:44398
I20260811 19:00:03.401988 15628 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58252:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 19:00:03.402438 16222 tablet_replica.cc:330] T 7c44d6635d3644bda8d1785cb3a9e148 P ff643294d6ac4617b8d13a10a12df80f: stopping tablet replica
W20260811 19:00:03.402503 15628 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 19:00:03.402963 16222 raft_consensus.cc:2229] T 7c44d6635d3644bda8d1785cb3a9e148 P ff643294d6ac4617b8d13a10a12df80f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:00:03.403457 16221 tablet_replica.cc:330] T 7c44d6635d3644bda8d1785cb3a9e148 P 223b20a2034147dcba72be0e4a12c28a: stopping tablet replica
I20260811 19:00:03.403779 16221 raft_consensus.cc:2229] T 7c44d6635d3644bda8d1785cb3a9e148 P 223b20a2034147dcba72be0e4a12c28a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:00:03.404003 16221 raft_consensus.cc:2258] T 7c44d6635d3644bda8d1785cb3a9e148 P 223b20a2034147dcba72be0e4a12c28a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:00:03.405516 16221 ts_tablet_manager.cc:1905] T 7c44d6635d3644bda8d1785cb3a9e148 P 223b20a2034147dcba72be0e4a12c28a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:00:03.409355 15775 tablet_service.cc:1467] Processing CreateTablet for tablet db87692985f8463facb9b4ab715db7c6 (DEFAULT_TABLE table=test-table [id=c846d85391184a6e95ac046f159711da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:00:03.409698 15775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db87692985f8463facb9b4ab715db7c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:00:03.409957 16221 ts_tablet_manager.cc:1918] T 7c44d6635d3644bda8d1785cb3a9e148 P 223b20a2034147dcba72be0e4a12c28a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 19:00:03.410068 16221 log.cc:1196] T 7c44d6635d3644bda8d1785cb3a9e148 P 223b20a2034147dcba72be0e4a12c28a: Deleting WAL directory at /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/minicluster-data/ts-1/wal/wals/7c44d6635d3644bda8d1785cb3a9e148
I20260811 19:00:03.410339 16076 tablet_service.cc:1467] Processing CreateTablet for tablet db87692985f8463facb9b4ab715db7c6 (DEFAULT_TABLE table=test-table [id=c846d85391184a6e95ac046f159711da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:00:03.410409 16221 ts_tablet_manager.cc:1939] T 7c44d6635d3644bda8d1785cb3a9e148 P 223b20a2034147dcba72be0e4a12c28a: Deleting consensus metadata
I20260811 19:00:03.410653 16076 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db87692985f8463facb9b4ab715db7c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:00:03.413798 16173 tablet_bootstrap.cc:492] T db87692985f8463facb9b4ab715db7c6 P ff643294d6ac4617b8d13a10a12df80f: Bootstrap starting.
I20260811 19:00:03.414961 16173 tablet_bootstrap.cc:654] T db87692985f8463facb9b4ab715db7c6 P ff643294d6ac4617b8d13a10a12df80f: Neither blocks nor log segments found. Creating new log.
I20260811 19:00:03.415895 15613 catalog_manager.cc:4841] TS 223b20a2034147dcba72be0e4a12c28a (127.13.109.2:40263): tablet 7c44d6635d3644bda8d1785cb3a9e148 (table test-table [id=10f3294d7f704a9eb3ffe11e7ffc9685]) successfully deleted
I20260811 19:00:03.416625 15926 tablet_service.cc:1467] Processing CreateTablet for tablet db87692985f8463facb9b4ab715db7c6 (DEFAULT_TABLE table=test-table [id=c846d85391184a6e95ac046f159711da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:00:03.416899 15926 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db87692985f8463facb9b4ab715db7c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:00:03.419421 16172 tablet_bootstrap.cc:492] T db87692985f8463facb9b4ab715db7c6 P 223b20a2034147dcba72be0e4a12c28a: Bootstrap starting.
I20260811 19:00:03.420435 16172 tablet_bootstrap.cc:654] T db87692985f8463facb9b4ab715db7c6 P 223b20a2034147dcba72be0e4a12c28a: Neither blocks nor log segments found. Creating new log.
I20260811 19:00:03.421752 16172 tablet_bootstrap.cc:492] T db87692985f8463facb9b4ab715db7c6 P 223b20a2034147dcba72be0e4a12c28a: No bootstrap required, opened a new log
I20260811 19:00:03.421895 16172 ts_tablet_manager.cc:1397] T db87692985f8463facb9b4ab715db7c6 P 223b20a2034147dcba72be0e4a12c28a: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 19:00:03.422353 16172 raft_consensus.cc:348] T db87692985f8463facb9b4ab715db7c6 P 223b20a2034147dcba72be0e4a12c28a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e08a2a95cd14e629ce94e2b5b75e6bd" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 43961 } } peers { permanent_uuid: "223b20a2034147dcba72be0e4a12c28a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 40263 } } peers { permanent_uuid: "ff643294d6ac4617b8d13a10a12df80f" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43919 } }
I20260811 19:00:03.422482 16172 raft_consensus.cc:374] T db87692985f8463facb9b4ab715db7c6 P 223b20a2034147dcba72be0e4a12c28a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:00:03.422513 16172 raft_consensus.cc:729] T db87692985f8463facb9b4ab715db7c6 P 223b20a2034147dcba72be0e4a12c28a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 223b20a2034147dcba72be0e4a12c28a, State: Initialized, Role: FOLLOWER
I20260811 19:00:03.422639 16172 consensus_queue.cc:260] T db87692985f8463facb9b4ab715db7c6 P 223b20a2034147dcba72be0e4a12c28a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e08a2a95cd14e629ce94e2b5b75e6bd" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 43961 } } peers { permanent_uuid: "223b20a2034147dcba72be0e4a12c28a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 40263 } } peers { permanent_uuid: "ff643294d6ac4617b8d13a10a12df80f" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43919 } }
I20260811 19:00:03.422911 16172 ts_tablet_manager.cc:1428] T db87692985f8463facb9b4ab715db7c6 P 223b20a2034147dcba72be0e4a12c28a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 19:00:03.424129 16171 tablet_bootstrap.cc:492] T db87692985f8463facb9b4ab715db7c6 P 9e08a2a95cd14e629ce94e2b5b75e6bd: Bootstrap starting.
I20260811 19:00:03.425117 16171 tablet_bootstrap.cc:654] T db87692985f8463facb9b4ab715db7c6 P 9e08a2a95cd14e629ce94e2b5b75e6bd: Neither blocks nor log segments found. Creating new log.
I20260811 19:00:03.426569 16173 tablet_bootstrap.cc:492] T db87692985f8463facb9b4ab715db7c6 P ff643294d6ac4617b8d13a10a12df80f: No bootstrap required, opened a new log
I20260811 19:00:03.426733 16173 ts_tablet_manager.cc:1397] T db87692985f8463facb9b4ab715db7c6 P ff643294d6ac4617b8d13a10a12df80f: Time spent bootstrapping tablet: real 0.013s user 0.000s sys 0.002s
I20260811 19:00:03.426900 16171 tablet_bootstrap.cc:492] T db87692985f8463facb9b4ab715db7c6 P 9e08a2a95cd14e629ce94e2b5b75e6bd: No bootstrap required, opened a new log
I20260811 19:00:03.426977 16171 ts_tablet_manager.cc:1397] T db87692985f8463facb9b4ab715db7c6 P 9e08a2a95cd14e629ce94e2b5b75e6bd: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260811 19:00:03.427235 16173 raft_consensus.cc:348] T db87692985f8463facb9b4ab715db7c6 P ff643294d6ac4617b8d13a10a12df80f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e08a2a95cd14e629ce94e2b5b75e6bd" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 43961 } } peers { permanent_uuid: "223b20a2034147dcba72be0e4a12c28a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 40263 } } peers { permanent_uuid: "ff643294d6ac4617b8d13a10a12df80f" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43919 } }
I20260811 19:00:03.427366 16173 raft_consensus.cc:374] T db87692985f8463facb9b4ab715db7c6 P ff643294d6ac4617b8d13a10a12df80f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:00:03.427395 16173 raft_consensus.cc:729] T db87692985f8463facb9b4ab715db7c6 P ff643294d6ac4617b8d13a10a12df80f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff643294d6ac4617b8d13a10a12df80f, State: Initialized, Role: FOLLOWER
I20260811 19:00:03.427393 16171 raft_consensus.cc:348] T db87692985f8463facb9b4ab715db7c6 P 9e08a2a95cd14e629ce94e2b5b75e6bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e08a2a95cd14e629ce94e2b5b75e6bd" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 43961 } } peers { permanent_uuid: "223b20a2034147dcba72be0e4a12c28a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 40263 } } peers { permanent_uuid: "ff643294d6ac4617b8d13a10a12df80f" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43919 } }
I20260811 19:00:03.427515 16171 raft_consensus.cc:374] T db87692985f8463facb9b4ab715db7c6 P 9e08a2a95cd14e629ce94e2b5b75e6bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:00:03.427546 16171 raft_consensus.cc:729] T db87692985f8463facb9b4ab715db7c6 P 9e08a2a95cd14e629ce94e2b5b75e6bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e08a2a95cd14e629ce94e2b5b75e6bd, State: Initialized, Role: FOLLOWER
I20260811 19:00:03.427676 16171 consensus_queue.cc:260] T db87692985f8463facb9b4ab715db7c6 P 9e08a2a95cd14e629ce94e2b5b75e6bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e08a2a95cd14e629ce94e2b5b75e6bd" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 43961 } } peers { permanent_uuid: "223b20a2034147dcba72be0e4a12c28a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 40263 } } peers { permanent_uuid: "ff643294d6ac4617b8d13a10a12df80f" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43919 } }
I20260811 19:00:03.427973 16171 ts_tablet_manager.cc:1428] T db87692985f8463facb9b4ab715db7c6 P 9e08a2a95cd14e629ce94e2b5b75e6bd: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 19:00:03.430361 16173 consensus_queue.cc:260] T db87692985f8463facb9b4ab715db7c6 P ff643294d6ac4617b8d13a10a12df80f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e08a2a95cd14e629ce94e2b5b75e6bd" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 43961 } } peers { permanent_uuid: "223b20a2034147dcba72be0e4a12c28a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 40263 } } peers { permanent_uuid: "ff643294d6ac4617b8d13a10a12df80f" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43919 } }
I20260811 19:00:03.430876 16173 ts_tablet_manager.cc:1428] T db87692985f8463facb9b4ab715db7c6 P ff643294d6ac4617b8d13a10a12df80f: Time spent starting tablet: real 0.004s user 0.000s sys 0.001s
I20260811 19:00:03.431095 16222 raft_consensus.cc:2258] T 7c44d6635d3644bda8d1785cb3a9e148 P ff643294d6ac4617b8d13a10a12df80f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:00:03.431514 16223 tablet_replica.cc:330] T 7c44d6635d3644bda8d1785cb3a9e148 P 9e08a2a95cd14e629ce94e2b5b75e6bd: stopping tablet replica
I20260811 19:00:03.432760 16222 ts_tablet_manager.cc:1905] T 7c44d6635d3644bda8d1785cb3a9e148 P ff643294d6ac4617b8d13a10a12df80f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:00:03.433223 16223 raft_consensus.cc:2229] T 7c44d6635d3644bda8d1785cb3a9e148 P 9e08a2a95cd14e629ce94e2b5b75e6bd [term 1 LEADER]: Raft consensus shutting down.
I20260811 19:00:03.436896 16222 ts_tablet_manager.cc:1918] T 7c44d6635d3644bda8d1785cb3a9e148 P ff643294d6ac4617b8d13a10a12df80f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 19:00:03.437409 16223 raft_consensus.cc:2258] T 7c44d6635d3644bda8d1785cb3a9e148 P 9e08a2a95cd14e629ce94e2b5b75e6bd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:00:03.437947 16222 log.cc:1196] T 7c44d6635d3644bda8d1785cb3a9e148 P ff643294d6ac4617b8d13a10a12df80f: Deleting WAL directory at /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/minicluster-data/ts-2/wal/wals/7c44d6635d3644bda8d1785cb3a9e148
I20260811 19:00:03.438323 16222 ts_tablet_manager.cc:1939] T 7c44d6635d3644bda8d1785cb3a9e148 P ff643294d6ac4617b8d13a10a12df80f: Deleting consensus metadata
I20260811 19:00:03.439141 16223 ts_tablet_manager.cc:1905] T 7c44d6635d3644bda8d1785cb3a9e148 P 9e08a2a95cd14e629ce94e2b5b75e6bd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:00:03.439671 15614 catalog_manager.cc:4841] TS ff643294d6ac4617b8d13a10a12df80f (127.13.109.3:43919): tablet 7c44d6635d3644bda8d1785cb3a9e148 (table test-table [id=10f3294d7f704a9eb3ffe11e7ffc9685]) successfully deleted
I20260811 19:00:03.443138 16223 ts_tablet_manager.cc:1918] T 7c44d6635d3644bda8d1785cb3a9e148 P 9e08a2a95cd14e629ce94e2b5b75e6bd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 19:00:03.443352 16223 log.cc:1196] T 7c44d6635d3644bda8d1785cb3a9e148 P 9e08a2a95cd14e629ce94e2b5b75e6bd: Deleting WAL directory at /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/minicluster-data/ts-0/wal/wals/7c44d6635d3644bda8d1785cb3a9e148
I20260811 19:00:03.443771 16223 ts_tablet_manager.cc:1939] T 7c44d6635d3644bda8d1785cb3a9e148 P 9e08a2a95cd14e629ce94e2b5b75e6bd: Deleting consensus metadata
I20260811 19:00:03.445199 15614 catalog_manager.cc:4841] TS 9e08a2a95cd14e629ce94e2b5b75e6bd (127.13.109.1:43961): tablet 7c44d6635d3644bda8d1785cb3a9e148 (table test-table [id=10f3294d7f704a9eb3ffe11e7ffc9685]) successfully deleted
I20260811 19:00:03.567823 16177 raft_consensus.cc:482] T db87692985f8463facb9b4ab715db7c6 P 9e08a2a95cd14e629ce94e2b5b75e6bd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:00:03.567993 16177 raft_consensus.cc:504] T db87692985f8463facb9b4ab715db7c6 P 9e08a2a95cd14e629ce94e2b5b75e6bd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e08a2a95cd14e629ce94e2b5b75e6bd" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 43961 } } peers { permanent_uuid: "223b20a2034147dcba72be0e4a12c28a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 40263 } } peers { permanent_uuid: "ff643294d6ac4617b8d13a10a12df80f" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43919 } }
I20260811 19:00:03.568518 16177 leader_election.cc:288] T db87692985f8463facb9b4ab715db7c6 P 9e08a2a95cd14e629ce94e2b5b75e6bd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 223b20a2034147dcba72be0e4a12c28a (127.13.109.2:40263), ff643294d6ac4617b8d13a10a12df80f (127.13.109.3:43919)
I20260811 19:00:03.569022 15946 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db87692985f8463facb9b4ab715db7c6" candidate_uuid: "9e08a2a95cd14e629ce94e2b5b75e6bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "223b20a2034147dcba72be0e4a12c28a" is_pre_election: true
I20260811 19:00:03.569079 16097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db87692985f8463facb9b4ab715db7c6" candidate_uuid: "9e08a2a95cd14e629ce94e2b5b75e6bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff643294d6ac4617b8d13a10a12df80f" is_pre_election: true
I20260811 19:00:03.569201 15946 raft_consensus.cc:2454] T db87692985f8463facb9b4ab715db7c6 P 223b20a2034147dcba72be0e4a12c28a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e08a2a95cd14e629ce94e2b5b75e6bd in term 0.
I20260811 19:00:03.569229 16097 raft_consensus.cc:2454] T db87692985f8463facb9b4ab715db7c6 P ff643294d6ac4617b8d13a10a12df80f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e08a2a95cd14e629ce94e2b5b75e6bd in term 0.
I20260811 19:00:03.569649 15725 leader_election.cc:302] T db87692985f8463facb9b4ab715db7c6 P 9e08a2a95cd14e629ce94e2b5b75e6bd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e08a2a95cd14e629ce94e2b5b75e6bd, ff643294d6ac4617b8d13a10a12df80f; no voters:
I20260811 19:00:03.569885 16177 raft_consensus.cc:2781] T db87692985f8463facb9b4ab715db7c6 P 9e08a2a95cd14e629ce94e2b5b75e6bd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 19:00:03.569965 16177 raft_consensus.cc:482] T db87692985f8463facb9b4ab715db7c6 P 9e08a2a95cd14e629ce94e2b5b75e6bd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 19:00:03.570024 16177 raft_consensus.cc:3037] T db87692985f8463facb9b4ab715db7c6 P 9e08a2a95cd14e629ce94e2b5b75e6bd [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:00:03.571144 16177 raft_consensus.cc:504] T db87692985f8463facb9b4ab715db7c6 P 9e08a2a95cd14e629ce94e2b5b75e6bd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e08a2a95cd14e629ce94e2b5b75e6bd" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 43961 } } peers { permanent_uuid: "223b20a2034147dcba72be0e4a12c28a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 40263 } } peers { permanent_uuid: "ff643294d6ac4617b8d13a10a12df80f" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43919 } }
I20260811 19:00:03.571640 16177 leader_election.cc:288] T db87692985f8463facb9b4ab715db7c6 P 9e08a2a95cd14e629ce94e2b5b75e6bd [CANDIDATE]: Term 1 election: Requested vote from peers 223b20a2034147dcba72be0e4a12c28a (127.13.109.2:40263), ff643294d6ac4617b8d13a10a12df80f (127.13.109.3:43919)
I20260811 19:00:03.572126 15946 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db87692985f8463facb9b4ab715db7c6" candidate_uuid: "9e08a2a95cd14e629ce94e2b5b75e6bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "223b20a2034147dcba72be0e4a12c28a"
I20260811 19:00:03.572119 16097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db87692985f8463facb9b4ab715db7c6" candidate_uuid: "9e08a2a95cd14e629ce94e2b5b75e6bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff643294d6ac4617b8d13a10a12df80f"
I20260811 19:00:03.572275 15946 raft_consensus.cc:3037] T db87692985f8463facb9b4ab715db7c6 P 223b20a2034147dcba72be0e4a12c28a [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:00:03.572275 16097 raft_consensus.cc:3037] T db87692985f8463facb9b4ab715db7c6 P ff643294d6ac4617b8d13a10a12df80f [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:00:03.573511 16097 raft_consensus.cc:2454] T db87692985f8463facb9b4ab715db7c6 P ff643294d6ac4617b8d13a10a12df80f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e08a2a95cd14e629ce94e2b5b75e6bd in term 1.
I20260811 19:00:03.573511 15946 raft_consensus.cc:2454] T db87692985f8463facb9b4ab715db7c6 P 223b20a2034147dcba72be0e4a12c28a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e08a2a95cd14e629ce94e2b5b75e6bd in term 1.
I20260811 19:00:03.573918 15724 leader_election.cc:302] T db87692985f8463facb9b4ab715db7c6 P 9e08a2a95cd14e629ce94e2b5b75e6bd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 223b20a2034147dcba72be0e4a12c28a, 9e08a2a95cd14e629ce94e2b5b75e6bd; no voters:
I20260811 19:00:03.574175 16177 raft_consensus.cc:2781] T db87692985f8463facb9b4ab715db7c6 P 9e08a2a95cd14e629ce94e2b5b75e6bd [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:00:03.574285 16177 raft_consensus.cc:686] T db87692985f8463facb9b4ab715db7c6 P 9e08a2a95cd14e629ce94e2b5b75e6bd [term 1 LEADER]: Becoming Leader. State: Replica: 9e08a2a95cd14e629ce94e2b5b75e6bd, State: Running, Role: LEADER
I20260811 19:00:03.574447 16177 consensus_queue.cc:237] T db87692985f8463facb9b4ab715db7c6 P 9e08a2a95cd14e629ce94e2b5b75e6bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e08a2a95cd14e629ce94e2b5b75e6bd" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 43961 } } peers { permanent_uuid: "223b20a2034147dcba72be0e4a12c28a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 40263 } } peers { permanent_uuid: "ff643294d6ac4617b8d13a10a12df80f" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43919 } }
I20260811 19:00:03.576400 15627 catalog_manager.cc:5495] T db87692985f8463facb9b4ab715db7c6 P 9e08a2a95cd14e629ce94e2b5b75e6bd reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e08a2a95cd14e629ce94e2b5b75e6bd (127.13.109.1). New cstate: current_term: 1 leader_uuid: "9e08a2a95cd14e629ce94e2b5b75e6bd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e08a2a95cd14e629ce94e2b5b75e6bd" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 43961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "223b20a2034147dcba72be0e4a12c28a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 40263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff643294d6ac4617b8d13a10a12df80f" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43919 } health_report { overall_health: UNKNOWN } } }
I20260811 19:00:03.598286 15946 raft_consensus.cc:1261] T db87692985f8463facb9b4ab715db7c6 P 223b20a2034147dcba72be0e4a12c28a [term 1 FOLLOWER]: Refusing update from remote peer 9e08a2a95cd14e629ce94e2b5b75e6bd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:00:03.598279 16097 raft_consensus.cc:1261] T db87692985f8463facb9b4ab715db7c6 P ff643294d6ac4617b8d13a10a12df80f [term 1 FOLLOWER]: Refusing update from remote peer 9e08a2a95cd14e629ce94e2b5b75e6bd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:00:03.598722 16177 consensus_queue.cc:1035] T db87692985f8463facb9b4ab715db7c6 P 9e08a2a95cd14e629ce94e2b5b75e6bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff643294d6ac4617b8d13a10a12df80f" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:00:03.598914 16216 consensus_queue.cc:1035] T db87692985f8463facb9b4ab715db7c6 P 9e08a2a95cd14e629ce94e2b5b75e6bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "223b20a2034147dcba72be0e4a12c28a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 40263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 19:00:03.616367 16201 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260811 19:00:05.184439 15628 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58252:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:00:05.184623 15628 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58252:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:00:05.187223 15628 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c5c0646e359b477aaf79adc6b3f50482: Sending DeleteTablet for 3 replicas of tablet db87692985f8463facb9b4ab715db7c6
I20260811 19:00:05.187846 15775 tablet_service.cc:1514] Processing DeleteTablet for tablet db87692985f8463facb9b4ab715db7c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:00:05 UTC) from {username='kudu', principal='kudu/127.13.109.62@KRBTEST.COM'} at 127.0.0.1:44398
I20260811 19:00:05.187894 15926 tablet_service.cc:1514] Processing DeleteTablet for tablet db87692985f8463facb9b4ab715db7c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:00:05 UTC) from {username='kudu', principal='kudu/127.13.109.62@KRBTEST.COM'} at 127.0.0.1:53002
I20260811 19:00:05.187920 16076 tablet_service.cc:1514] Processing DeleteTablet for tablet db87692985f8463facb9b4ab715db7c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:00:05 UTC) from {username='kudu', principal='kudu/127.13.109.62@KRBTEST.COM'} at 127.0.0.1:36028
I20260811 19:00:05.188246 16254 tablet_replica.cc:330] T db87692985f8463facb9b4ab715db7c6 P 9e08a2a95cd14e629ce94e2b5b75e6bd: stopping tablet replica
I20260811 19:00:05.188367 16255 tablet_replica.cc:330] T db87692985f8463facb9b4ab715db7c6 P 223b20a2034147dcba72be0e4a12c28a: stopping tablet replica
I20260811 19:00:05.188397 16254 raft_consensus.cc:2229] T db87692985f8463facb9b4ab715db7c6 P 9e08a2a95cd14e629ce94e2b5b75e6bd [term 1 LEADER]: Raft consensus shutting down.
I20260811 19:00:05.188558 16255 raft_consensus.cc:2229] T db87692985f8463facb9b4ab715db7c6 P 223b20a2034147dcba72be0e4a12c28a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:00:05.188616 16254 raft_consensus.cc:2258] T db87692985f8463facb9b4ab715db7c6 P 9e08a2a95cd14e629ce94e2b5b75e6bd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:00:05.188958 16255 raft_consensus.cc:2258] T db87692985f8463facb9b4ab715db7c6 P 223b20a2034147dcba72be0e4a12c28a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:00:05.189512 16254 ts_tablet_manager.cc:1905] T db87692985f8463facb9b4ab715db7c6 P 9e08a2a95cd14e629ce94e2b5b75e6bd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:00:05.190564 16256 tablet_replica.cc:330] T db87692985f8463facb9b4ab715db7c6 P ff643294d6ac4617b8d13a10a12df80f: stopping tablet replica
I20260811 19:00:05.190697 16256 raft_consensus.cc:2229] T db87692985f8463facb9b4ab715db7c6 P ff643294d6ac4617b8d13a10a12df80f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:00:05.190819 16256 raft_consensus.cc:2258] T db87692985f8463facb9b4ab715db7c6 P ff643294d6ac4617b8d13a10a12df80f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:00:05.191387 16255 ts_tablet_manager.cc:1905] T db87692985f8463facb9b4ab715db7c6 P 223b20a2034147dcba72be0e4a12c28a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:00:05.191987 16256 ts_tablet_manager.cc:1905] T db87692985f8463facb9b4ab715db7c6 P ff643294d6ac4617b8d13a10a12df80f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:00:05.192619 16254 ts_tablet_manager.cc:1918] T db87692985f8463facb9b4ab715db7c6 P 9e08a2a95cd14e629ce94e2b5b75e6bd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 19:00:05.192726 16254 log.cc:1196] T db87692985f8463facb9b4ab715db7c6 P 9e08a2a95cd14e629ce94e2b5b75e6bd: Deleting WAL directory at /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/minicluster-data/ts-0/wal/wals/db87692985f8463facb9b4ab715db7c6
I20260811 19:00:05.193050 16254 ts_tablet_manager.cc:1939] T db87692985f8463facb9b4ab715db7c6 P 9e08a2a95cd14e629ce94e2b5b75e6bd: Deleting consensus metadata
I20260811 19:00:05.194065 16255 ts_tablet_manager.cc:1918] T db87692985f8463facb9b4ab715db7c6 P 223b20a2034147dcba72be0e4a12c28a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 19:00:05.194150 16255 log.cc:1196] T db87692985f8463facb9b4ab715db7c6 P 223b20a2034147dcba72be0e4a12c28a: Deleting WAL directory at /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/minicluster-data/ts-1/wal/wals/db87692985f8463facb9b4ab715db7c6
I20260811 19:00:05.194109 15614 catalog_manager.cc:4841] TS 9e08a2a95cd14e629ce94e2b5b75e6bd (127.13.109.1:43961): tablet db87692985f8463facb9b4ab715db7c6 (table test-table [id=c846d85391184a6e95ac046f159711da]) successfully deleted
I20260811 19:00:05.194429 16255 ts_tablet_manager.cc:1939] T db87692985f8463facb9b4ab715db7c6 P 223b20a2034147dcba72be0e4a12c28a: Deleting consensus metadata
I20260811 19:00:05.195178 15613 catalog_manager.cc:4841] TS 223b20a2034147dcba72be0e4a12c28a (127.13.109.2:40263): tablet db87692985f8463facb9b4ab715db7c6 (table test-table [id=c846d85391184a6e95ac046f159711da]) successfully deleted
I20260811 19:00:05.195782 16256 ts_tablet_manager.cc:1918] T db87692985f8463facb9b4ab715db7c6 P ff643294d6ac4617b8d13a10a12df80f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 19:00:05.195879 16256 log.cc:1196] T db87692985f8463facb9b4ab715db7c6 P ff643294d6ac4617b8d13a10a12df80f: Deleting WAL directory at /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/minicluster-data/ts-2/wal/wals/db87692985f8463facb9b4ab715db7c6
I20260811 19:00:05.196197 16256 ts_tablet_manager.cc:1939] T db87692985f8463facb9b4ab715db7c6 P ff643294d6ac4617b8d13a10a12df80f: Deleting consensus metadata
I20260811 19:00:05.196918 15614 catalog_manager.cc:4841] TS ff643294d6ac4617b8d13a10a12df80f (127.13.109.3:43919): tablet db87692985f8463facb9b4ab715db7c6 (table test-table [id=c846d85391184a6e95ac046f159711da]) successfully deleted
Aug 11 19:00:05 dist-test-slave-0kls krb5kdc[15582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474805, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 19:00:05 dist-test-slave-0kls krb5kdc[15582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474805, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
I20260811 19:00:05.219336 16266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:05.210282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58272 (local address 127.13.109.62:34491)
0811 19:00:05.210632 (+ 350us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:05.210643 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:05.210660 (+ 17us) server_negotiation.cc:407] Connection header received
0811 19:00:05.210806 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:05.210810 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:05.210868 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:05.210966 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:05.211706 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:05.212275 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:05.213137 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:05.213380 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:05.213520 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:05.216017 (+ 2497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:05.216020 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:05.216022 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:05.216049 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:05.217655 (+ 1606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:05.218095 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:05.218097 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:05.218098 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:05.218176 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:05.218435 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:05.218437 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:05.218438 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:05.218625 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:05.218780 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:05.219080 (+ 300us) server_negotiation.cc:299] Negotiation successful
0811 19:00:05.219192 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":259,"thread_start_us":131,"threads_started":1}
I20260811 19:00:05.222208 15628 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58272:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 19:00:05.222693 15628 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 19:00:05.227468 15775 tablet_service.cc:1467] Processing CreateTablet for tablet c6918d8fe5f34f49b14d5150b7879e77 (DEFAULT_TABLE table=test-table [id=9f1e56a6802b4aa2994c460182daf9be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:00:05.227828 15775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6918d8fe5f34f49b14d5150b7879e77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:00:05.227788 16076 tablet_service.cc:1467] Processing CreateTablet for tablet c6918d8fe5f34f49b14d5150b7879e77 (DEFAULT_TABLE table=test-table [id=9f1e56a6802b4aa2994c460182daf9be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:00:05.228055 16076 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6918d8fe5f34f49b14d5150b7879e77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:00:05.230150 15926 tablet_service.cc:1467] Processing CreateTablet for tablet c6918d8fe5f34f49b14d5150b7879e77 (DEFAULT_TABLE table=test-table [id=9f1e56a6802b4aa2994c460182daf9be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:00:05.230460 15926 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6918d8fe5f34f49b14d5150b7879e77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:00:05.230839 16267 tablet_bootstrap.cc:492] T c6918d8fe5f34f49b14d5150b7879e77 P ff643294d6ac4617b8d13a10a12df80f: Bootstrap starting.
I20260811 19:00:05.231588 16267 tablet_bootstrap.cc:654] T c6918d8fe5f34f49b14d5150b7879e77 P ff643294d6ac4617b8d13a10a12df80f: Neither blocks nor log segments found. Creating new log.
I20260811 19:00:05.232141 16268 tablet_bootstrap.cc:492] T c6918d8fe5f34f49b14d5150b7879e77 P 9e08a2a95cd14e629ce94e2b5b75e6bd: Bootstrap starting.
I20260811 19:00:05.232551 16270 tablet_bootstrap.cc:492] T c6918d8fe5f34f49b14d5150b7879e77 P 223b20a2034147dcba72be0e4a12c28a: Bootstrap starting.
I20260811 19:00:05.232795 16267 tablet_bootstrap.cc:492] T c6918d8fe5f34f49b14d5150b7879e77 P ff643294d6ac4617b8d13a10a12df80f: No bootstrap required, opened a new log
I20260811 19:00:05.232884 16267 ts_tablet_manager.cc:1397] T c6918d8fe5f34f49b14d5150b7879e77 P ff643294d6ac4617b8d13a10a12df80f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 19:00:05.233094 16268 tablet_bootstrap.cc:654] T c6918d8fe5f34f49b14d5150b7879e77 P 9e08a2a95cd14e629ce94e2b5b75e6bd: Neither blocks nor log segments found. Creating new log.
I20260811 19:00:05.233394 16267 raft_consensus.cc:348] T c6918d8fe5f34f49b14d5150b7879e77 P ff643294d6ac4617b8d13a10a12df80f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e08a2a95cd14e629ce94e2b5b75e6bd" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 43961 } } peers { permanent_uuid: "223b20a2034147dcba72be0e4a12c28a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 40263 } } peers { permanent_uuid: "ff643294d6ac4617b8d13a10a12df80f" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43919 } }
I20260811 19:00:05.233522 16267 raft_consensus.cc:374] T c6918d8fe5f34f49b14d5150b7879e77 P ff643294d6ac4617b8d13a10a12df80f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:00:05.233573 16267 raft_consensus.cc:729] T c6918d8fe5f34f49b14d5150b7879e77 P ff643294d6ac4617b8d13a10a12df80f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff643294d6ac4617b8d13a10a12df80f, State: Initialized, Role: FOLLOWER
I20260811 19:00:05.233729 16267 consensus_queue.cc:260] T c6918d8fe5f34f49b14d5150b7879e77 P ff643294d6ac4617b8d13a10a12df80f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e08a2a95cd14e629ce94e2b5b75e6bd" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 43961 } } peers { permanent_uuid: "223b20a2034147dcba72be0e4a12c28a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 40263 } } peers { permanent_uuid: "ff643294d6ac4617b8d13a10a12df80f" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43919 } }
I20260811 19:00:05.234107 16267 ts_tablet_manager.cc:1428] T c6918d8fe5f34f49b14d5150b7879e77 P ff643294d6ac4617b8d13a10a12df80f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 19:00:05.234225 16270 tablet_bootstrap.cc:654] T c6918d8fe5f34f49b14d5150b7879e77 P 223b20a2034147dcba72be0e4a12c28a: Neither blocks nor log segments found. Creating new log.
I20260811 19:00:05.234405 16268 tablet_bootstrap.cc:492] T c6918d8fe5f34f49b14d5150b7879e77 P 9e08a2a95cd14e629ce94e2b5b75e6bd: No bootstrap required, opened a new log
I20260811 19:00:05.234529 16268 ts_tablet_manager.cc:1397] T c6918d8fe5f34f49b14d5150b7879e77 P 9e08a2a95cd14e629ce94e2b5b75e6bd: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 19:00:05.234963 16268 raft_consensus.cc:348] T c6918d8fe5f34f49b14d5150b7879e77 P 9e08a2a95cd14e629ce94e2b5b75e6bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e08a2a95cd14e629ce94e2b5b75e6bd" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 43961 } } peers { permanent_uuid: "223b20a2034147dcba72be0e4a12c28a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 40263 } } peers { permanent_uuid: "ff643294d6ac4617b8d13a10a12df80f" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43919 } }
I20260811 19:00:05.235116 16268 raft_consensus.cc:374] T c6918d8fe5f34f49b14d5150b7879e77 P 9e08a2a95cd14e629ce94e2b5b75e6bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:00:05.235174 16268 raft_consensus.cc:729] T c6918d8fe5f34f49b14d5150b7879e77 P 9e08a2a95cd14e629ce94e2b5b75e6bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e08a2a95cd14e629ce94e2b5b75e6bd, State: Initialized, Role: FOLLOWER
I20260811 19:00:05.235306 16268 consensus_queue.cc:260] T c6918d8fe5f34f49b14d5150b7879e77 P 9e08a2a95cd14e629ce94e2b5b75e6bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e08a2a95cd14e629ce94e2b5b75e6bd" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 43961 } } peers { permanent_uuid: "223b20a2034147dcba72be0e4a12c28a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 40263 } } peers { permanent_uuid: "ff643294d6ac4617b8d13a10a12df80f" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43919 } }
I20260811 19:00:05.235603 16268 ts_tablet_manager.cc:1428] T c6918d8fe5f34f49b14d5150b7879e77 P 9e08a2a95cd14e629ce94e2b5b75e6bd: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 19:00:05.235620 16270 tablet_bootstrap.cc:492] T c6918d8fe5f34f49b14d5150b7879e77 P 223b20a2034147dcba72be0e4a12c28a: No bootstrap required, opened a new log
I20260811 19:00:05.235710 16270 ts_tablet_manager.cc:1397] T c6918d8fe5f34f49b14d5150b7879e77 P 223b20a2034147dcba72be0e4a12c28a: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 19:00:05.236384 16270 raft_consensus.cc:348] T c6918d8fe5f34f49b14d5150b7879e77 P 223b20a2034147dcba72be0e4a12c28a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e08a2a95cd14e629ce94e2b5b75e6bd" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 43961 } } peers { permanent_uuid: "223b20a2034147dcba72be0e4a12c28a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 40263 } } peers { permanent_uuid: "ff643294d6ac4617b8d13a10a12df80f" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43919 } }
I20260811 19:00:05.236505 16270 raft_consensus.cc:374] T c6918d8fe5f34f49b14d5150b7879e77 P 223b20a2034147dcba72be0e4a12c28a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:00:05.236544 16270 raft_consensus.cc:729] T c6918d8fe5f34f49b14d5150b7879e77 P 223b20a2034147dcba72be0e4a12c28a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 223b20a2034147dcba72be0e4a12c28a, State: Initialized, Role: FOLLOWER
I20260811 19:00:05.236708 16270 consensus_queue.cc:260] T c6918d8fe5f34f49b14d5150b7879e77 P 223b20a2034147dcba72be0e4a12c28a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e08a2a95cd14e629ce94e2b5b75e6bd" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 43961 } } peers { permanent_uuid: "223b20a2034147dcba72be0e4a12c28a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 40263 } } peers { permanent_uuid: "ff643294d6ac4617b8d13a10a12df80f" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43919 } }
I20260811 19:00:05.237200 16270 ts_tablet_manager.cc:1428] T c6918d8fe5f34f49b14d5150b7879e77 P 223b20a2034147dcba72be0e4a12c28a: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260811 19:00:05.273119 16273 raft_consensus.cc:482] T c6918d8fe5f34f49b14d5150b7879e77 P 223b20a2034147dcba72be0e4a12c28a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:00:05.273299 16273 raft_consensus.cc:504] T c6918d8fe5f34f49b14d5150b7879e77 P 223b20a2034147dcba72be0e4a12c28a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e08a2a95cd14e629ce94e2b5b75e6bd" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 43961 } } peers { permanent_uuid: "223b20a2034147dcba72be0e4a12c28a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 40263 } } peers { permanent_uuid: "ff643294d6ac4617b8d13a10a12df80f" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43919 } }
I20260811 19:00:05.274523 16273 leader_election.cc:288] T c6918d8fe5f34f49b14d5150b7879e77 P 223b20a2034147dcba72be0e4a12c28a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e08a2a95cd14e629ce94e2b5b75e6bd (127.13.109.1:43961), ff643294d6ac4617b8d13a10a12df80f (127.13.109.3:43919)
I20260811 19:00:05.279898 16275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:05.274954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.2:51489 (local address 127.13.109.1:43961)
0811 19:00:05.275303 (+ 349us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:05.275313 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:05.275335 (+ 22us) server_negotiation.cc:407] Connection header received
0811 19:00:05.275570 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:05.275575 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:05.275643 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:05.275905 (+ 262us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:00:05.276296 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:05.277104 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:05.278572 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:05.279204 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:05.279336 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:05.279398 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:05.279641 (+ 243us) server_negotiation.cc:299] Negotiation successful
0811 19:00:05.279747 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":245,"thread_start_us":147,"threads_started":1}
I20260811 19:00:05.279906 16276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:05.275327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.3:43919 (local address 127.13.109.2:59895)
0811 19:00:05.276009 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:05.276029 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:05.276136 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:05.276494 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:05.276497 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:05.276514 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:00:05.276652 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:05.276658 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:05.277432 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:05.277435 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:05.278450 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:05.278459 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:05.279372 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:05.279375 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:05.279531 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:05.279549 (+ 18us) client_negotiation.cc:769] Sending connection context
0811 19:00:05.279653 (+ 104us) client_negotiation.cc:240] Negotiation successful
0811 19:00:05.279741 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":396,"mutex_wait_us":34,"thread_start_us":136,"threads_started":1}
I20260811 19:00:05.279932 16274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:05.274764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.1:43961 (local address 127.13.109.2:51489)
0811 19:00:05.275151 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:05.275169 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:05.275289 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:05.276012 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:05.276019 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:05.276045 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:00:05.276161 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:05.276167 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:05.277261 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:05.277267 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:05.278419 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:05.278431 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:05.279341 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:05.279345 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:05.279460 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:05.279521 (+ 61us) client_negotiation.cc:769] Sending connection context
0811 19:00:05.279591 (+ 70us) client_negotiation.cc:240] Negotiation successful
0811 19:00:05.279711 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"mutex_wait_us":13,"thread_start_us":171,"threads_started":1}
I20260811 19:00:05.279942 16277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:05.275641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.2:59895 (local address 127.13.109.3:43919)
0811 19:00:05.276145 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:05.276238 (+ 93us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:05.276268 (+ 30us) server_negotiation.cc:407] Connection header received
0811 19:00:05.276331 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:05.276334 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:05.276388 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:05.276480 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:00:05.276771 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:05.277292 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:05.278620 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:05.279218 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:05.279372 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:05.279436 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:05.279689 (+ 253us) server_negotiation.cc:299] Negotiation successful
0811 19:00:05.279794 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":220,"threads_started":1}
I20260811 19:00:05.280519 15795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6918d8fe5f34f49b14d5150b7879e77" candidate_uuid: "223b20a2034147dcba72be0e4a12c28a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e08a2a95cd14e629ce94e2b5b75e6bd" is_pre_election: true
I20260811 19:00:05.280720 15795 raft_consensus.cc:2454] T c6918d8fe5f34f49b14d5150b7879e77 P 9e08a2a95cd14e629ce94e2b5b75e6bd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 223b20a2034147dcba72be0e4a12c28a in term 0.
I20260811 19:00:05.280838 16097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6918d8fe5f34f49b14d5150b7879e77" candidate_uuid: "223b20a2034147dcba72be0e4a12c28a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff643294d6ac4617b8d13a10a12df80f" is_pre_election: true
I20260811 19:00:05.281008 16097 raft_consensus.cc:2454] T c6918d8fe5f34f49b14d5150b7879e77 P ff643294d6ac4617b8d13a10a12df80f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 223b20a2034147dcba72be0e4a12c28a in term 0.
I20260811 19:00:05.281250 15876 leader_election.cc:302] T c6918d8fe5f34f49b14d5150b7879e77 P 223b20a2034147dcba72be0e4a12c28a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 223b20a2034147dcba72be0e4a12c28a, 9e08a2a95cd14e629ce94e2b5b75e6bd; no voters:
I20260811 19:00:05.281627 16273 raft_consensus.cc:2781] T c6918d8fe5f34f49b14d5150b7879e77 P 223b20a2034147dcba72be0e4a12c28a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 19:00:05.281692 16273 raft_consensus.cc:482] T c6918d8fe5f34f49b14d5150b7879e77 P 223b20a2034147dcba72be0e4a12c28a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 19:00:05.281719 16273 raft_consensus.cc:3037] T c6918d8fe5f34f49b14d5150b7879e77 P 223b20a2034147dcba72be0e4a12c28a [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:00:05.282840 16273 raft_consensus.cc:504] T c6918d8fe5f34f49b14d5150b7879e77 P 223b20a2034147dcba72be0e4a12c28a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e08a2a95cd14e629ce94e2b5b75e6bd" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 43961 } } peers { permanent_uuid: "223b20a2034147dcba72be0e4a12c28a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 40263 } } peers { permanent_uuid: "ff643294d6ac4617b8d13a10a12df80f" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43919 } }
I20260811 19:00:05.283174 16273 leader_election.cc:288] T c6918d8fe5f34f49b14d5150b7879e77 P 223b20a2034147dcba72be0e4a12c28a [CANDIDATE]: Term 1 election: Requested vote from peers 9e08a2a95cd14e629ce94e2b5b75e6bd (127.13.109.1:43961), ff643294d6ac4617b8d13a10a12df80f (127.13.109.3:43919)
I20260811 19:00:05.283612 15795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6918d8fe5f34f49b14d5150b7879e77" candidate_uuid: "223b20a2034147dcba72be0e4a12c28a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e08a2a95cd14e629ce94e2b5b75e6bd"
I20260811 19:00:05.283782 15795 raft_consensus.cc:3037] T c6918d8fe5f34f49b14d5150b7879e77 P 9e08a2a95cd14e629ce94e2b5b75e6bd [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:00:05.283726 16097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6918d8fe5f34f49b14d5150b7879e77" candidate_uuid: "223b20a2034147dcba72be0e4a12c28a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff643294d6ac4617b8d13a10a12df80f"
I20260811 19:00:05.283875 16097 raft_consensus.cc:3037] T c6918d8fe5f34f49b14d5150b7879e77 P ff643294d6ac4617b8d13a10a12df80f [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:00:05.284870 15795 raft_consensus.cc:2454] T c6918d8fe5f34f49b14d5150b7879e77 P 9e08a2a95cd14e629ce94e2b5b75e6bd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 223b20a2034147dcba72be0e4a12c28a in term 1.
I20260811 19:00:05.284893 16097 raft_consensus.cc:2454] T c6918d8fe5f34f49b14d5150b7879e77 P ff643294d6ac4617b8d13a10a12df80f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 223b20a2034147dcba72be0e4a12c28a in term 1.
I20260811 19:00:05.285207 15876 leader_election.cc:302] T c6918d8fe5f34f49b14d5150b7879e77 P 223b20a2034147dcba72be0e4a12c28a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 223b20a2034147dcba72be0e4a12c28a, ff643294d6ac4617b8d13a10a12df80f; no voters:
I20260811 19:00:05.285452 16273 raft_consensus.cc:2781] T c6918d8fe5f34f49b14d5150b7879e77 P 223b20a2034147dcba72be0e4a12c28a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:00:05.285742 16273 raft_consensus.cc:686] T c6918d8fe5f34f49b14d5150b7879e77 P 223b20a2034147dcba72be0e4a12c28a [term 1 LEADER]: Becoming Leader. State: Replica: 223b20a2034147dcba72be0e4a12c28a, State: Running, Role: LEADER
I20260811 19:00:05.286067 16273 consensus_queue.cc:237] T c6918d8fe5f34f49b14d5150b7879e77 P 223b20a2034147dcba72be0e4a12c28a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e08a2a95cd14e629ce94e2b5b75e6bd" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 43961 } } peers { permanent_uuid: "223b20a2034147dcba72be0e4a12c28a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 40263 } } peers { permanent_uuid: "ff643294d6ac4617b8d13a10a12df80f" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43919 } }
I20260811 19:00:05.288542 15628 catalog_manager.cc:5495] T c6918d8fe5f34f49b14d5150b7879e77 P 223b20a2034147dcba72be0e4a12c28a reported cstate change: term changed from 0 to 1, leader changed from <none> to 223b20a2034147dcba72be0e4a12c28a (127.13.109.2). New cstate: current_term: 1 leader_uuid: "223b20a2034147dcba72be0e4a12c28a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e08a2a95cd14e629ce94e2b5b75e6bd" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 43961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "223b20a2034147dcba72be0e4a12c28a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 40263 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff643294d6ac4617b8d13a10a12df80f" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43919 } health_report { overall_health: UNKNOWN } } }
I20260811 19:00:05.306015 16281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:05.301948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53012 (local address 127.13.109.2:40263)
0811 19:00:05.302296 (+ 348us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:05.302305 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:05.302317 (+ 12us) server_negotiation.cc:407] Connection header received
0811 19:00:05.302402 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:05.302404 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:05.302459 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:05.302592 (+ 133us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:00:05.302869 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:05.303444 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:05.304339 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:05.304582 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:05.304699 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:05.305014 (+ 315us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:00:05.305528 (+ 514us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:00:05.305675 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:05.305740 (+ 65us) server_negotiation.cc:299] Negotiation successful
0811 19:00:05.305832 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266,"thread_start_us":130,"threads_started":1}
I20260811 19:00:05.316967 16277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:05.313679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.2:54975 (local address 127.13.109.3:43919)
0811 19:00:05.313860 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:05.313865 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:05.314084 (+ 219us) server_negotiation.cc:407] Connection header received
0811 19:00:05.314193 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:05.314195 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:05.314245 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:05.314310 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:00:05.314545 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:05.315067 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:05.315895 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:05.316120 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:05.316204 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:05.316478 (+ 274us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:00:05.316554 (+ 76us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:00:05.316614 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:05.316738 (+ 124us) server_negotiation.cc:299] Negotiation successful
0811 19:00:05.316817 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 19:00:05.316978 16284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:05.313577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.3:43919 (local address 127.13.109.2:54975)
0811 19:00:05.313996 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:05.314010 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:05.314107 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:05.314335 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:05.314337 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:05.314344 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 19:00:05.314450 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:05.314454 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:05.315163 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:05.315165 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:05.315774 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:05.315782 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:05.316228 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:05.316230 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:05.316299 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:05.316350 (+ 51us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 19:00:05.316628 (+ 278us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 19:00:05.316632 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 19:00:05.316734 (+ 102us) client_negotiation.cc:240] Negotiation successful
0811 19:00:05.316845 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":139,"threads_started":1}
I20260811 19:00:05.323884 16097 raft_consensus.cc:1261] T c6918d8fe5f34f49b14d5150b7879e77 P ff643294d6ac4617b8d13a10a12df80f [term 1 FOLLOWER]: Refusing update from remote peer 223b20a2034147dcba72be0e4a12c28a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:00:05.323884 15795 raft_consensus.cc:1261] T c6918d8fe5f34f49b14d5150b7879e77 P 9e08a2a95cd14e629ce94e2b5b75e6bd [term 1 FOLLOWER]: Refusing update from remote peer 223b20a2034147dcba72be0e4a12c28a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:00:05.324429 16278 consensus_queue.cc:1035] T c6918d8fe5f34f49b14d5150b7879e77 P 223b20a2034147dcba72be0e4a12c28a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff643294d6ac4617b8d13a10a12df80f" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:00:05.324664 16273 consensus_queue.cc:1035] T c6918d8fe5f34f49b14d5150b7879e77 P 223b20a2034147dcba72be0e4a12c28a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e08a2a95cd14e629ce94e2b5b75e6bd" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 43961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:00:05.342980 16281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:05.339965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.3:53887 (local address 127.13.109.2:40263)
0811 19:00:05.340072 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:05.340075 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:05.340087 (+ 12us) server_negotiation.cc:407] Connection header received
0811 19:00:05.340144 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:05.340146 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:05.340186 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:05.340246 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:00:05.340581 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:05.341078 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:05.341919 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:05.342146 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:05.342225 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:05.342517 (+ 292us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:00:05.342599 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:00:05.342697 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:05.342793 (+ 96us) server_negotiation.cc:299] Negotiation successful
0811 19:00:05.342842 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260811 19:00:05.342994 16290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:05.339635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.2:40263 (local address 127.13.109.3:53887)
0811 19:00:05.339951 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:05.339966 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:05.340067 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:05.340284 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:05.340287 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:05.340300 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 19:00:05.340425 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:05.340431 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:05.341180 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:05.341183 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:05.341812 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:05.341821 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:05.342260 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:05.342263 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:05.342347 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:05.342375 (+ 28us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 19:00:05.342709 (+ 334us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 19:00:05.342719 (+ 10us) client_negotiation.cc:769] Sending connection context
0811 19:00:05.342783 (+ 64us) client_negotiation.cc:240] Negotiation successful
0811 19:00:05.342868 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":146,"threads_started":1}
I20260811 19:00:05.369716 15626 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58272:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:00:05.369966 15626 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58272:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:00:05.372417 15626 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c5c0646e359b477aaf79adc6b3f50482: Sending DeleteTablet for 3 replicas of tablet c6918d8fe5f34f49b14d5150b7879e77
I20260811 19:00:05.373210 15775 tablet_service.cc:1514] Processing DeleteTablet for tablet c6918d8fe5f34f49b14d5150b7879e77 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:00:05 UTC) from {username='kudu', principal='kudu/127.13.109.62@KRBTEST.COM'} at 127.0.0.1:44398
I20260811 19:00:05.373298 16076 tablet_service.cc:1514] Processing DeleteTablet for tablet c6918d8fe5f34f49b14d5150b7879e77 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:00:05 UTC) from {username='kudu', principal='kudu/127.13.109.62@KRBTEST.COM'} at 127.0.0.1:36028
I20260811 19:00:05.373313 15926 tablet_service.cc:1514] Processing DeleteTablet for tablet c6918d8fe5f34f49b14d5150b7879e77 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:00:05 UTC) from {username='kudu', principal='kudu/127.13.109.62@KRBTEST.COM'} at 127.0.0.1:53002
I20260811 19:00:05.373544 16254 tablet_replica.cc:330] T c6918d8fe5f34f49b14d5150b7879e77 P 9e08a2a95cd14e629ce94e2b5b75e6bd: stopping tablet replica
I20260811 19:00:05.373582 16256 tablet_replica.cc:330] T c6918d8fe5f34f49b14d5150b7879e77 P ff643294d6ac4617b8d13a10a12df80f: stopping tablet replica
I20260811 19:00:05.373692 16254 raft_consensus.cc:2229] T c6918d8fe5f34f49b14d5150b7879e77 P 9e08a2a95cd14e629ce94e2b5b75e6bd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:00:05.373778 16256 raft_consensus.cc:2229] T c6918d8fe5f34f49b14d5150b7879e77 P ff643294d6ac4617b8d13a10a12df80f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:00:05.373898 16255 tablet_replica.cc:330] T c6918d8fe5f34f49b14d5150b7879e77 P 223b20a2034147dcba72be0e4a12c28a: stopping tablet replica
I20260811 19:00:05.373943 16256 raft_consensus.cc:2258] T c6918d8fe5f34f49b14d5150b7879e77 P ff643294d6ac4617b8d13a10a12df80f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:00:05.374014 16255 raft_consensus.cc:2229] T c6918d8fe5f34f49b14d5150b7879e77 P 223b20a2034147dcba72be0e4a12c28a [term 1 LEADER]: Raft consensus shutting down.
I20260811 19:00:05.374377 16255 raft_consensus.cc:2258] T c6918d8fe5f34f49b14d5150b7879e77 P 223b20a2034147dcba72be0e4a12c28a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:00:05.374567 16254 raft_consensus.cc:2258] T c6918d8fe5f34f49b14d5150b7879e77 P 9e08a2a95cd14e629ce94e2b5b75e6bd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:00:05.374862 16256 ts_tablet_manager.cc:1905] T c6918d8fe5f34f49b14d5150b7879e77 P ff643294d6ac4617b8d13a10a12df80f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:00:05.375320 16255 ts_tablet_manager.cc:1905] T c6918d8fe5f34f49b14d5150b7879e77 P 223b20a2034147dcba72be0e4a12c28a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:00:05.377197 16256 ts_tablet_manager.cc:1918] T c6918d8fe5f34f49b14d5150b7879e77 P ff643294d6ac4617b8d13a10a12df80f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 19:00:05.377300 16256 log.cc:1196] T c6918d8fe5f34f49b14d5150b7879e77 P ff643294d6ac4617b8d13a10a12df80f: Deleting WAL directory at /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/minicluster-data/ts-2/wal/wals/c6918d8fe5f34f49b14d5150b7879e77
I20260811 19:00:05.377552 16256 ts_tablet_manager.cc:1939] T c6918d8fe5f34f49b14d5150b7879e77 P ff643294d6ac4617b8d13a10a12df80f: Deleting consensus metadata
I20260811 19:00:05.377940 16255 ts_tablet_manager.cc:1918] T c6918d8fe5f34f49b14d5150b7879e77 P 223b20a2034147dcba72be0e4a12c28a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 19:00:05.378019 16255 log.cc:1196] T c6918d8fe5f34f49b14d5150b7879e77 P 223b20a2034147dcba72be0e4a12c28a: Deleting WAL directory at /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/minicluster-data/ts-1/wal/wals/c6918d8fe5f34f49b14d5150b7879e77
I20260811 19:00:05.378286 16255 ts_tablet_manager.cc:1939] T c6918d8fe5f34f49b14d5150b7879e77 P 223b20a2034147dcba72be0e4a12c28a: Deleting consensus metadata
I20260811 19:00:05.378407 15614 catalog_manager.cc:4841] TS ff643294d6ac4617b8d13a10a12df80f (127.13.109.3:43919): tablet c6918d8fe5f34f49b14d5150b7879e77 (table test-table [id=9f1e56a6802b4aa2994c460182daf9be]) successfully deleted
I20260811 19:00:05.378737 16254 ts_tablet_manager.cc:1905] T c6918d8fe5f34f49b14d5150b7879e77 P 9e08a2a95cd14e629ce94e2b5b75e6bd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:00:05.378998 15613 catalog_manager.cc:4841] TS 223b20a2034147dcba72be0e4a12c28a (127.13.109.2:40263): tablet c6918d8fe5f34f49b14d5150b7879e77 (table test-table [id=9f1e56a6802b4aa2994c460182daf9be]) successfully deleted
I20260811 19:00:05.379789 15626 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58272:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 19:00:05.380287 15626 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 19:00:05.381398 16254 ts_tablet_manager.cc:1918] T c6918d8fe5f34f49b14d5150b7879e77 P 9e08a2a95cd14e629ce94e2b5b75e6bd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 19:00:05.381491 16254 log.cc:1196] T c6918d8fe5f34f49b14d5150b7879e77 P 9e08a2a95cd14e629ce94e2b5b75e6bd: Deleting WAL directory at /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/minicluster-data/ts-0/wal/wals/c6918d8fe5f34f49b14d5150b7879e77
I20260811 19:00:05.381773 16254 ts_tablet_manager.cc:1939] T c6918d8fe5f34f49b14d5150b7879e77 P 9e08a2a95cd14e629ce94e2b5b75e6bd: Deleting consensus metadata
I20260811 19:00:05.382853 15614 catalog_manager.cc:4841] TS 9e08a2a95cd14e629ce94e2b5b75e6bd (127.13.109.1:43961): tablet c6918d8fe5f34f49b14d5150b7879e77 (table test-table [id=9f1e56a6802b4aa2994c460182daf9be]) successfully deleted
I20260811 19:00:05.385669 15775 tablet_service.cc:1467] Processing CreateTablet for tablet e59eb0699fe24aaf8d6296b1dc735677 (DEFAULT_TABLE table=test-table [id=69ca2f35bf57498fa3c017ffe3fb0ed5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:00:05.386006 15775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e59eb0699fe24aaf8d6296b1dc735677. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:00:05.386005 15926 tablet_service.cc:1467] Processing CreateTablet for tablet e59eb0699fe24aaf8d6296b1dc735677 (DEFAULT_TABLE table=test-table [id=69ca2f35bf57498fa3c017ffe3fb0ed5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:00:05.386188 16076 tablet_service.cc:1467] Processing CreateTablet for tablet e59eb0699fe24aaf8d6296b1dc735677 (DEFAULT_TABLE table=test-table [id=69ca2f35bf57498fa3c017ffe3fb0ed5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:00:05.386294 15926 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e59eb0699fe24aaf8d6296b1dc735677. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:00:05.386458 16076 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e59eb0699fe24aaf8d6296b1dc735677. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:00:05.388664 16268 tablet_bootstrap.cc:492] T e59eb0699fe24aaf8d6296b1dc735677 P 9e08a2a95cd14e629ce94e2b5b75e6bd: Bootstrap starting.
I20260811 19:00:05.389103 16267 tablet_bootstrap.cc:492] T e59eb0699fe24aaf8d6296b1dc735677 P ff643294d6ac4617b8d13a10a12df80f: Bootstrap starting.
I20260811 19:00:05.389566 16268 tablet_bootstrap.cc:654] T e59eb0699fe24aaf8d6296b1dc735677 P 9e08a2a95cd14e629ce94e2b5b75e6bd: Neither blocks nor log segments found. Creating new log.
I20260811 19:00:05.389921 16267 tablet_bootstrap.cc:654] T e59eb0699fe24aaf8d6296b1dc735677 P ff643294d6ac4617b8d13a10a12df80f: Neither blocks nor log segments found. Creating new log.
I20260811 19:00:05.390791 16268 tablet_bootstrap.cc:492] T e59eb0699fe24aaf8d6296b1dc735677 P 9e08a2a95cd14e629ce94e2b5b75e6bd: No bootstrap required, opened a new log
I20260811 19:00:05.390890 16268 ts_tablet_manager.cc:1397] T e59eb0699fe24aaf8d6296b1dc735677 P 9e08a2a95cd14e629ce94e2b5b75e6bd: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260811 19:00:05.391321 16268 raft_consensus.cc:348] T e59eb0699fe24aaf8d6296b1dc735677 P 9e08a2a95cd14e629ce94e2b5b75e6bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e08a2a95cd14e629ce94e2b5b75e6bd" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 43961 } } peers { permanent_uuid: "223b20a2034147dcba72be0e4a12c28a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 40263 } } peers { permanent_uuid: "ff643294d6ac4617b8d13a10a12df80f" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43919 } }
I20260811 19:00:05.391450 16268 raft_consensus.cc:374] T e59eb0699fe24aaf8d6296b1dc735677 P 9e08a2a95cd14e629ce94e2b5b75e6bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:00:05.391482 16268 raft_consensus.cc:729] T e59eb0699fe24aaf8d6296b1dc735677 P 9e08a2a95cd14e629ce94e2b5b75e6bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e08a2a95cd14e629ce94e2b5b75e6bd, State: Initialized, Role: FOLLOWER
I20260811 19:00:05.391619 16268 consensus_queue.cc:260] T e59eb0699fe24aaf8d6296b1dc735677 P 9e08a2a95cd14e629ce94e2b5b75e6bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e08a2a95cd14e629ce94e2b5b75e6bd" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 43961 } } peers { permanent_uuid: "223b20a2034147dcba72be0e4a12c28a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 40263 } } peers { permanent_uuid: "ff643294d6ac4617b8d13a10a12df80f" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43919 } }
I20260811 19:00:05.391893 16268 ts_tablet_manager.cc:1428] T e59eb0699fe24aaf8d6296b1dc735677 P 9e08a2a95cd14e629ce94e2b5b75e6bd: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260811 19:00:05.392308 16270 tablet_bootstrap.cc:492] T e59eb0699fe24aaf8d6296b1dc735677 P 223b20a2034147dcba72be0e4a12c28a: Bootstrap starting.
I20260811 19:00:05.394042 16270 tablet_bootstrap.cc:654] T e59eb0699fe24aaf8d6296b1dc735677 P 223b20a2034147dcba72be0e4a12c28a: Neither blocks nor log segments found. Creating new log.
I20260811 19:00:05.394358 16267 tablet_bootstrap.cc:492] T e59eb0699fe24aaf8d6296b1dc735677 P ff643294d6ac4617b8d13a10a12df80f: No bootstrap required, opened a new log
I20260811 19:00:05.394464 16267 ts_tablet_manager.cc:1397] T e59eb0699fe24aaf8d6296b1dc735677 P ff643294d6ac4617b8d13a10a12df80f: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20260811 19:00:05.394893 16267 raft_consensus.cc:348] T e59eb0699fe24aaf8d6296b1dc735677 P ff643294d6ac4617b8d13a10a12df80f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e08a2a95cd14e629ce94e2b5b75e6bd" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 43961 } } peers { permanent_uuid: "223b20a2034147dcba72be0e4a12c28a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 40263 } } peers { permanent_uuid: "ff643294d6ac4617b8d13a10a12df80f" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43919 } }
I20260811 19:00:05.395031 16267 raft_consensus.cc:374] T e59eb0699fe24aaf8d6296b1dc735677 P ff643294d6ac4617b8d13a10a12df80f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:00:05.395077 16267 raft_consensus.cc:729] T e59eb0699fe24aaf8d6296b1dc735677 P ff643294d6ac4617b8d13a10a12df80f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff643294d6ac4617b8d13a10a12df80f, State: Initialized, Role: FOLLOWER
I20260811 19:00:05.395222 16267 consensus_queue.cc:260] T e59eb0699fe24aaf8d6296b1dc735677 P ff643294d6ac4617b8d13a10a12df80f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e08a2a95cd14e629ce94e2b5b75e6bd" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 43961 } } peers { permanent_uuid: "223b20a2034147dcba72be0e4a12c28a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 40263 } } peers { permanent_uuid: "ff643294d6ac4617b8d13a10a12df80f" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43919 } }
I20260811 19:00:05.395426 16270 tablet_bootstrap.cc:492] T e59eb0699fe24aaf8d6296b1dc735677 P 223b20a2034147dcba72be0e4a12c28a: No bootstrap required, opened a new log
I20260811 19:00:05.395509 16270 ts_tablet_manager.cc:1397] T e59eb0699fe24aaf8d6296b1dc735677 P 223b20a2034147dcba72be0e4a12c28a: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 19:00:05.395514 16267 ts_tablet_manager.cc:1428] T e59eb0699fe24aaf8d6296b1dc735677 P ff643294d6ac4617b8d13a10a12df80f: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 19:00:05.396023 16270 raft_consensus.cc:348] T e59eb0699fe24aaf8d6296b1dc735677 P 223b20a2034147dcba72be0e4a12c28a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e08a2a95cd14e629ce94e2b5b75e6bd" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 43961 } } peers { permanent_uuid: "223b20a2034147dcba72be0e4a12c28a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 40263 } } peers { permanent_uuid: "ff643294d6ac4617b8d13a10a12df80f" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43919 } }
I20260811 19:00:05.396148 16270 raft_consensus.cc:374] T e59eb0699fe24aaf8d6296b1dc735677 P 223b20a2034147dcba72be0e4a12c28a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:00:05.396186 16270 raft_consensus.cc:729] T e59eb0699fe24aaf8d6296b1dc735677 P 223b20a2034147dcba72be0e4a12c28a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 223b20a2034147dcba72be0e4a12c28a, State: Initialized, Role: FOLLOWER
I20260811 19:00:05.396862 16270 consensus_queue.cc:260] T e59eb0699fe24aaf8d6296b1dc735677 P 223b20a2034147dcba72be0e4a12c28a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e08a2a95cd14e629ce94e2b5b75e6bd" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 43961 } } peers { permanent_uuid: "223b20a2034147dcba72be0e4a12c28a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 40263 } } peers { permanent_uuid: "ff643294d6ac4617b8d13a10a12df80f" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43919 } }
I20260811 19:00:05.397184 16270 ts_tablet_manager.cc:1428] T e59eb0699fe24aaf8d6296b1dc735677 P 223b20a2034147dcba72be0e4a12c28a: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260811 19:00:05.403684 16230 raft_consensus.cc:482] T e59eb0699fe24aaf8d6296b1dc735677 P 9e08a2a95cd14e629ce94e2b5b75e6bd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:00:05.403868 16230 raft_consensus.cc:504] T e59eb0699fe24aaf8d6296b1dc735677 P 9e08a2a95cd14e629ce94e2b5b75e6bd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e08a2a95cd14e629ce94e2b5b75e6bd" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 43961 } } peers { permanent_uuid: "223b20a2034147dcba72be0e4a12c28a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 40263 } } peers { permanent_uuid: "ff643294d6ac4617b8d13a10a12df80f" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43919 } }
I20260811 19:00:05.404512 16230 leader_election.cc:288] T e59eb0699fe24aaf8d6296b1dc735677 P 9e08a2a95cd14e629ce94e2b5b75e6bd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 223b20a2034147dcba72be0e4a12c28a (127.13.109.2:40263), ff643294d6ac4617b8d13a10a12df80f (127.13.109.3:43919)
I20260811 19:00:05.405007 15946 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e59eb0699fe24aaf8d6296b1dc735677" candidate_uuid: "9e08a2a95cd14e629ce94e2b5b75e6bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "223b20a2034147dcba72be0e4a12c28a" is_pre_election: true
I20260811 19:00:05.405047 16097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e59eb0699fe24aaf8d6296b1dc735677" candidate_uuid: "9e08a2a95cd14e629ce94e2b5b75e6bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff643294d6ac4617b8d13a10a12df80f" is_pre_election: true
I20260811 19:00:05.405200 16097 raft_consensus.cc:2454] T e59eb0699fe24aaf8d6296b1dc735677 P ff643294d6ac4617b8d13a10a12df80f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e08a2a95cd14e629ce94e2b5b75e6bd in term 0.
I20260811 19:00:05.405200 15946 raft_consensus.cc:2454] T e59eb0699fe24aaf8d6296b1dc735677 P 223b20a2034147dcba72be0e4a12c28a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e08a2a95cd14e629ce94e2b5b75e6bd in term 0.
I20260811 19:00:05.405544 15725 leader_election.cc:302] T e59eb0699fe24aaf8d6296b1dc735677 P 9e08a2a95cd14e629ce94e2b5b75e6bd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e08a2a95cd14e629ce94e2b5b75e6bd, ff643294d6ac4617b8d13a10a12df80f; no voters:
I20260811 19:00:05.405928 16230 raft_consensus.cc:2781] T e59eb0699fe24aaf8d6296b1dc735677 P 9e08a2a95cd14e629ce94e2b5b75e6bd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 19:00:05.405997 16230 raft_consensus.cc:482] T e59eb0699fe24aaf8d6296b1dc735677 P 9e08a2a95cd14e629ce94e2b5b75e6bd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 19:00:05.406034 16230 raft_consensus.cc:3037] T e59eb0699fe24aaf8d6296b1dc735677 P 9e08a2a95cd14e629ce94e2b5b75e6bd [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:00:05.407141 16230 raft_consensus.cc:504] T e59eb0699fe24aaf8d6296b1dc735677 P 9e08a2a95cd14e629ce94e2b5b75e6bd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e08a2a95cd14e629ce94e2b5b75e6bd" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 43961 } } peers { permanent_uuid: "223b20a2034147dcba72be0e4a12c28a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 40263 } } peers { permanent_uuid: "ff643294d6ac4617b8d13a10a12df80f" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43919 } }
I20260811 19:00:05.407631 16230 leader_election.cc:288] T e59eb0699fe24aaf8d6296b1dc735677 P 9e08a2a95cd14e629ce94e2b5b75e6bd [CANDIDATE]: Term 1 election: Requested vote from peers 223b20a2034147dcba72be0e4a12c28a (127.13.109.2:40263), ff643294d6ac4617b8d13a10a12df80f (127.13.109.3:43919)
I20260811 19:00:05.408382 16097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e59eb0699fe24aaf8d6296b1dc735677" candidate_uuid: "9e08a2a95cd14e629ce94e2b5b75e6bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff643294d6ac4617b8d13a10a12df80f"
I20260811 19:00:05.408382 15946 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e59eb0699fe24aaf8d6296b1dc735677" candidate_uuid: "9e08a2a95cd14e629ce94e2b5b75e6bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "223b20a2034147dcba72be0e4a12c28a"
I20260811 19:00:05.408553 15946 raft_consensus.cc:3037] T e59eb0699fe24aaf8d6296b1dc735677 P 223b20a2034147dcba72be0e4a12c28a [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:00:05.408553 16097 raft_consensus.cc:3037] T e59eb0699fe24aaf8d6296b1dc735677 P ff643294d6ac4617b8d13a10a12df80f [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:00:05.409940 16097 raft_consensus.cc:2454] T e59eb0699fe24aaf8d6296b1dc735677 P ff643294d6ac4617b8d13a10a12df80f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e08a2a95cd14e629ce94e2b5b75e6bd in term 1.
I20260811 19:00:05.409940 15946 raft_consensus.cc:2454] T e59eb0699fe24aaf8d6296b1dc735677 P 223b20a2034147dcba72be0e4a12c28a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e08a2a95cd14e629ce94e2b5b75e6bd in term 1.
I20260811 19:00:05.410319 15724 leader_election.cc:302] T e59eb0699fe24aaf8d6296b1dc735677 P 9e08a2a95cd14e629ce94e2b5b75e6bd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 223b20a2034147dcba72be0e4a12c28a, 9e08a2a95cd14e629ce94e2b5b75e6bd; no voters:
I20260811 19:00:05.410537 16230 raft_consensus.cc:2781] T e59eb0699fe24aaf8d6296b1dc735677 P 9e08a2a95cd14e629ce94e2b5b75e6bd [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:00:05.410669 16230 raft_consensus.cc:686] T e59eb0699fe24aaf8d6296b1dc735677 P 9e08a2a95cd14e629ce94e2b5b75e6bd [term 1 LEADER]: Becoming Leader. State: Replica: 9e08a2a95cd14e629ce94e2b5b75e6bd, State: Running, Role: LEADER
I20260811 19:00:05.410844 16230 consensus_queue.cc:237] T e59eb0699fe24aaf8d6296b1dc735677 P 9e08a2a95cd14e629ce94e2b5b75e6bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e08a2a95cd14e629ce94e2b5b75e6bd" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 43961 } } peers { permanent_uuid: "223b20a2034147dcba72be0e4a12c28a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 40263 } } peers { permanent_uuid: "ff643294d6ac4617b8d13a10a12df80f" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43919 } }
I20260811 19:00:05.412833 15626 catalog_manager.cc:5495] T e59eb0699fe24aaf8d6296b1dc735677 P 9e08a2a95cd14e629ce94e2b5b75e6bd reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e08a2a95cd14e629ce94e2b5b75e6bd (127.13.109.1). New cstate: current_term: 1 leader_uuid: "9e08a2a95cd14e629ce94e2b5b75e6bd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e08a2a95cd14e629ce94e2b5b75e6bd" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 43961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "223b20a2034147dcba72be0e4a12c28a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 40263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff643294d6ac4617b8d13a10a12df80f" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43919 } health_report { overall_health: UNKNOWN } } }
I20260811 19:00:05.435178 16275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:05.431662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44414 (local address 127.13.109.1:43961)
0811 19:00:05.431791 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:05.431795 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:05.431808 (+ 13us) server_negotiation.cc:407] Connection header received
0811 19:00:05.431949 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:05.431951 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:05.431994 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:05.432061 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:00:05.432340 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:05.432844 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:05.434048 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:05.434277 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:05.434353 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:05.434658 (+ 305us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:00:05.434734 (+ 76us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:00:05.434815 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:05.434914 (+ 99us) server_negotiation.cc:299] Negotiation successful
0811 19:00:05.434971 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 19:00:05.442984 15946 raft_consensus.cc:1261] T e59eb0699fe24aaf8d6296b1dc735677 P 223b20a2034147dcba72be0e4a12c28a [term 1 FOLLOWER]: Refusing update from remote peer 9e08a2a95cd14e629ce94e2b5b75e6bd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:00:05.443287 16097 raft_consensus.cc:1261] T e59eb0699fe24aaf8d6296b1dc735677 P ff643294d6ac4617b8d13a10a12df80f [term 1 FOLLOWER]: Refusing update from remote peer 9e08a2a95cd14e629ce94e2b5b75e6bd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:00:05.443507 16245 consensus_queue.cc:1035] T e59eb0699fe24aaf8d6296b1dc735677 P 9e08a2a95cd14e629ce94e2b5b75e6bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "223b20a2034147dcba72be0e4a12c28a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 40263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:00:05.444337 16245 consensus_queue.cc:1035] T e59eb0699fe24aaf8d6296b1dc735677 P 9e08a2a95cd14e629ce94e2b5b75e6bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff643294d6ac4617b8d13a10a12df80f" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 19:00:05.460794 16234 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260811 19:00:07.024008 15626 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58272:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:00:07.024312 15626 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58272:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:00:07.027019 15626 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c5c0646e359b477aaf79adc6b3f50482: Sending DeleteTablet for 3 replicas of tablet e59eb0699fe24aaf8d6296b1dc735677
I20260811 19:00:07.027638 15775 tablet_service.cc:1514] Processing DeleteTablet for tablet e59eb0699fe24aaf8d6296b1dc735677 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:00:07 UTC) from {username='kudu', principal='kudu/127.13.109.62@KRBTEST.COM'} at 127.0.0.1:44398
I20260811 19:00:07.027799 15926 tablet_service.cc:1514] Processing DeleteTablet for tablet e59eb0699fe24aaf8d6296b1dc735677 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:00:07 UTC) from {username='kudu', principal='kudu/127.13.109.62@KRBTEST.COM'} at 127.0.0.1:53002
I20260811 19:00:07.028036 16076 tablet_service.cc:1514] Processing DeleteTablet for tablet e59eb0699fe24aaf8d6296b1dc735677 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:00:07 UTC) from {username='kudu', principal='kudu/127.13.109.62@KRBTEST.COM'} at 127.0.0.1:36028
I20260811 19:00:07.028087 16323 tablet_replica.cc:330] T e59eb0699fe24aaf8d6296b1dc735677 P 9e08a2a95cd14e629ce94e2b5b75e6bd: stopping tablet replica
I20260811 19:00:07.028223 16323 raft_consensus.cc:2229] T e59eb0699fe24aaf8d6296b1dc735677 P 9e08a2a95cd14e629ce94e2b5b75e6bd [term 1 LEADER]: Raft consensus shutting down.
I20260811 19:00:07.028250 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 15705
I20260811 19:00:07.028463 16323 raft_consensus.cc:2258] T e59eb0699fe24aaf8d6296b1dc735677 P 9e08a2a95cd14e629ce94e2b5b75e6bd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:00:07.030820 16325 tablet_replica.cc:330] T e59eb0699fe24aaf8d6296b1dc735677 P ff643294d6ac4617b8d13a10a12df80f: stopping tablet replica
I20260811 19:00:07.031095 16325 raft_consensus.cc:2229] T e59eb0699fe24aaf8d6296b1dc735677 P ff643294d6ac4617b8d13a10a12df80f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:00:07.031158 16324 tablet_replica.cc:330] T e59eb0699fe24aaf8d6296b1dc735677 P 223b20a2034147dcba72be0e4a12c28a: stopping tablet replica
I20260811 19:00:07.031298 16325 raft_consensus.cc:2258] T e59eb0699fe24aaf8d6296b1dc735677 P ff643294d6ac4617b8d13a10a12df80f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:00:07.031333 16324 raft_consensus.cc:2229] T e59eb0699fe24aaf8d6296b1dc735677 P 223b20a2034147dcba72be0e4a12c28a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:00:07.031497 16324 raft_consensus.cc:2258] T e59eb0699fe24aaf8d6296b1dc735677 P 223b20a2034147dcba72be0e4a12c28a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:00:07.032155 16324 ts_tablet_manager.cc:1905] T e59eb0699fe24aaf8d6296b1dc735677 P 223b20a2034147dcba72be0e4a12c28a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:00:07.032171 16325 ts_tablet_manager.cc:1905] T e59eb0699fe24aaf8d6296b1dc735677 P ff643294d6ac4617b8d13a10a12df80f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:00:07.034735 16324 ts_tablet_manager.cc:1918] T e59eb0699fe24aaf8d6296b1dc735677 P 223b20a2034147dcba72be0e4a12c28a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 19:00:07.034838 16324 log.cc:1196] T e59eb0699fe24aaf8d6296b1dc735677 P 223b20a2034147dcba72be0e4a12c28a: Deleting WAL directory at /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/minicluster-data/ts-1/wal/wals/e59eb0699fe24aaf8d6296b1dc735677
I20260811 19:00:07.035090 16324 ts_tablet_manager.cc:1939] T e59eb0699fe24aaf8d6296b1dc735677 P 223b20a2034147dcba72be0e4a12c28a: Deleting consensus metadata
I20260811 19:00:07.035795 16325 ts_tablet_manager.cc:1918] T e59eb0699fe24aaf8d6296b1dc735677 P ff643294d6ac4617b8d13a10a12df80f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 19:00:07.035884 16325 log.cc:1196] T e59eb0699fe24aaf8d6296b1dc735677 P ff643294d6ac4617b8d13a10a12df80f: Deleting WAL directory at /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474792496492-13748-0/minicluster-data/ts-2/wal/wals/e59eb0699fe24aaf8d6296b1dc735677
I20260811 19:00:07.035965 15613 catalog_manager.cc:4841] TS 223b20a2034147dcba72be0e4a12c28a (127.13.109.2:40263): tablet e59eb0699fe24aaf8d6296b1dc735677 (table test-table [id=69ca2f35bf57498fa3c017ffe3fb0ed5]) successfully deleted
I20260811 19:00:07.036180 16325 ts_tablet_manager.cc:1939] T e59eb0699fe24aaf8d6296b1dc735677 P ff643294d6ac4617b8d13a10a12df80f: Deleting consensus metadata
I20260811 19:00:07.037103 15614 catalog_manager.cc:4841] TS ff643294d6ac4617b8d13a10a12df80f (127.13.109.3:43919): tablet e59eb0699fe24aaf8d6296b1dc735677 (table test-table [id=69ca2f35bf57498fa3c017ffe3fb0ed5]) successfully deleted
W20260811 19:00:07.040675 15614 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.109.1:43961 (error 108)
I20260811 19:00:07.040918 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 15853
I20260811 19:00:07.045254 16327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:07.044747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.1:43961 (local address 127.0.0.1:44416)
0811 19:00:07.045068 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:07.045150 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.1:43961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":112,"threads_started":1}
W20260811 19:00:07.045606 15614 catalog_manager.cc:4584] TS 9e08a2a95cd14e629ce94e2b5b75e6bd (127.13.109.1:43961): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e59eb0699fe24aaf8d6296b1dc735677: Network error: Client connection negotiation failed: client connection to 127.13.109.1:43961: connect: Connection refused (error 111)
I20260811 19:00:07.051784 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 16007
I20260811 19:00:07.062600 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 15595
2026-08-11T19:00:07Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6504 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:00:07 dist-test-slave-0kls krb5kdc[16330](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:00:07 dist-test-slave-0kls krb5kdc[16330](info): set up 2 sockets
Aug 11 19:00:07 dist-test-slave-0kls krb5kdc[16330](info): commencing operation
krb5kdc: starting...
W20260811 19:00:08.174898 13748 mini_kdc.cc:121] Time spent starting KDC: real 1.086s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/krb5kdc/test-user.keytab.
Aug 11 19:00:08 dist-test-slave-0kls krb5kdc[16330](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474808, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:00:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:00:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.109.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.109.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.109.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab.
Entry for principal kudu/127.13.109.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab.
I20260811 19:00:08.309314 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.62:43929
--webserver_interface=127.13.109.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab
--principal=kudu/127.13.109.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:42795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.109.62:43929
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:00:08.435241 16344 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:00:08.435707 16344 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:00:08.435806 16344 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:00:08.442368 16344 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:00:08.442445 16344 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:00:08.442466 16344 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:00:08.442484 16344 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:00:08.442500 16344 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:00:08.448365 16344 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:42795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.109.62:43929
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab
--principal=kudu/127.13.109.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.62:43929
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.109.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16344
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:00:08.449579 16344 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:00:08.450803 16344 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:00:08.460302 16352 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:08.460319 16354 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:08.460321 16351 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:08.460584 16344 server_base.cc:1034] running on GCE node
I20260811 19:00:08.461468 16344 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:00:08.462911 16344 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:00:08.464152 16344 hybrid_clock.cc:648] HybridClock initialized: now 1786474808464130 us; error 42 us; skew 500 ppm
Aug 11 19:00:08 dist-test-slave-0kls krb5kdc[16330](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474808, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:00:08.467386 16344 init.cc:377] Logged in from keytab as kudu/127.13.109.62@KRBTEST.COM (short username kudu)
I20260811 19:00:08.468564 16344 webserver.cc:460] Webserver started at http://127.13.109.62:43117/ using document root <none> and password file <none>
I20260811 19:00:08.469211 16344 fs_manager.cc:359] Metadata directory not provided
I20260811 19:00:08.469285 16344 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:00:08.469484 16344 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:00:08.471944 16344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/minicluster-data/master-0/data/instance:
uuid: "5f92db9b8bb447b3ae758416783268c6"
format_stamp: "Formatted at 2026-08-11 19:00:08 on dist-test-slave-0kls"
server_key: "1cb2b2040ad5c1159760b1696e4f0f38"
server_key_iv: "02fa3e905f4eb84bf256fb3268e2725c"
server_key_version: "encryptionkey@0"
I20260811 19:00:08.472522 16344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/minicluster-data/master-0/wal/instance:
uuid: "5f92db9b8bb447b3ae758416783268c6"
format_stamp: "Formatted at 2026-08-11 19:00:08 on dist-test-slave-0kls"
server_key: "1cb2b2040ad5c1159760b1696e4f0f38"
server_key_iv: "02fa3e905f4eb84bf256fb3268e2725c"
server_key_version: "encryptionkey@0"
I20260811 19:00:08.476871 16344 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260811 19:00:08.479890 16361 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:08.481475 16344 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 19:00:08.481626 16344 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/minicluster-data/master-0/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/minicluster-data/master-0/wal
uuid: "5f92db9b8bb447b3ae758416783268c6"
format_stamp: "Formatted at 2026-08-11 19:00:08 on dist-test-slave-0kls"
server_key: "1cb2b2040ad5c1159760b1696e4f0f38"
server_key_iv: "02fa3e905f4eb84bf256fb3268e2725c"
server_key_version: "encryptionkey@0"
I20260811 19:00:08.481748 16344 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:00:08.498018 16344 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:00:08.502137 16344 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:00:08.502331 16344 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:00:08.510171 16344 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.62:43929
I20260811 19:00:08.510190 16417 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.62:43929 every 8 connection(s)
I20260811 19:00:08.511270 16344 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
I20260811 19:00:08.514437 16418 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:00:08.518605 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 16344
I20260811 19:00:08.518743 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/minicluster-data/master-0/wal/instance
I20260811 19:00:08.519101 13748 external_mini_cluster.cc:1428] Setting key 3698982e20ffeb3fbd4a9b4344652512
I20260811 19:00:08.521427 16418 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f92db9b8bb447b3ae758416783268c6: Bootstrap starting.
I20260811 19:00:08.523921 16418 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5f92db9b8bb447b3ae758416783268c6: Neither blocks nor log segments found. Creating new log.
I20260811 19:00:08.524744 16418 log.cc:824] T 00000000000000000000000000000000 P 5f92db9b8bb447b3ae758416783268c6: Log is configured to *not* fsync() on all Append() calls
I20260811 19:00:08.526973 16418 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f92db9b8bb447b3ae758416783268c6: No bootstrap required, opened a new log
Aug 11 19:00:08 dist-test-slave-0kls krb5kdc[16330](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474808, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
I20260811 19:00:08.531077 16418 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5f92db9b8bb447b3ae758416783268c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f92db9b8bb447b3ae758416783268c6" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 43929 } }
I20260811 19:00:08.531411 16418 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5f92db9b8bb447b3ae758416783268c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:00:08.531474 16418 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5f92db9b8bb447b3ae758416783268c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f92db9b8bb447b3ae758416783268c6, State: Initialized, Role: FOLLOWER
I20260811 19:00:08.532045 16418 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5f92db9b8bb447b3ae758416783268c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f92db9b8bb447b3ae758416783268c6" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 43929 } }
I20260811 19:00:08.532217 16418 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5f92db9b8bb447b3ae758416783268c6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:00:08.532303 16418 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5f92db9b8bb447b3ae758416783268c6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:00:08.532410 16418 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5f92db9b8bb447b3ae758416783268c6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:00:08.533741 16418 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5f92db9b8bb447b3ae758416783268c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f92db9b8bb447b3ae758416783268c6" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 43929 } }
I20260811 19:00:08.534202 16418 leader_election.cc:302] T 00000000000000000000000000000000 P 5f92db9b8bb447b3ae758416783268c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5f92db9b8bb447b3ae758416783268c6; no voters:
I20260811 19:00:08.534637 16418 leader_election.cc:288] T 00000000000000000000000000000000 P 5f92db9b8bb447b3ae758416783268c6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:00:08.534773 16423 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5f92db9b8bb447b3ae758416783268c6 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:00:08.535048 16423 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5f92db9b8bb447b3ae758416783268c6 [term 1 LEADER]: Becoming Leader. State: Replica: 5f92db9b8bb447b3ae758416783268c6, State: Running, Role: LEADER
I20260811 19:00:08.535396 16423 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5f92db9b8bb447b3ae758416783268c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f92db9b8bb447b3ae758416783268c6" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 43929 } }
I20260811 19:00:08.535784 16421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:08.520801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39800 (local address 127.13.109.62:43929)
0811 19:00:08.521333 (+ 532us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:08.521349 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:08.521389 (+ 40us) server_negotiation.cc:407] Connection header received
0811 19:00:08.522228 (+ 839us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:08.522249 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:08.522620 (+ 371us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:08.522987 (+ 367us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:08.523879 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:08.525025 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:08.525997 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:08.526480 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:08.526661 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:08.529235 (+ 2574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:08.529242 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:08.529263 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:08.529297 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:08.531934 (+ 2637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:08.532658 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:08.532667 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:08.532677 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:08.532838 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:08.533200 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:08.533203 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:08.533205 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:08.533868 (+ 663us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:08.534087 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:08.534369 (+ 282us) server_negotiation.cc:299] Negotiation successful
0811 19:00:08.534723 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":164,"threads_started":1}
I20260811 19:00:08.536091 16418 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5f92db9b8bb447b3ae758416783268c6 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:00:08.537442 16424 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f92db9b8bb447b3ae758416783268c6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5f92db9b8bb447b3ae758416783268c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f92db9b8bb447b3ae758416783268c6" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 43929 } } }
I20260811 19:00:08.537611 16424 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f92db9b8bb447b3ae758416783268c6 [sys.catalog]: This master's current role is: LEADER
I20260811 19:00:08.537442 16425 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f92db9b8bb447b3ae758416783268c6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5f92db9b8bb447b3ae758416783268c6. Latest consensus state: current_term: 1 leader_uuid: "5f92db9b8bb447b3ae758416783268c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f92db9b8bb447b3ae758416783268c6" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 43929 } } }
I20260811 19:00:08.537734 16425 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f92db9b8bb447b3ae758416783268c6 [sys.catalog]: This master's current role is: LEADER
I20260811 19:00:08.538808 16432 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:00:08.542238 16432 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:00:08.549042 16432 catalog_manager.cc:1347] Generated new cluster ID: 28d4dd8d58fa4be7871ba7b23c796fea
I20260811 19:00:08.549137 16432 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:00:08.560501 16432 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:00:08.561957 16432 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:00:08.574508 16432 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5f92db9b8bb447b3ae758416783268c6: Generated new TSK 0
I20260811 19:00:08.575522 16432 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.109.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.109.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.109.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.1.keytab.
Entry for principal kudu/127.13.109.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.1.keytab.
I20260811 19:00:08.616938 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.1:0
--local_ip_for_outbound_sockets=127.13.109.1
--webserver_interface=127.13.109.1
--webserver_port=0
--tserver_master_addrs=127.13.109.62:43929
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.1.keytab
--principal=kudu/127.13.109.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:42795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:00:08.742208 16447 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:00:08.742514 16447 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:00:08.742561 16447 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:00:08.747543 16447 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:00:08.747625 16447 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:00:08.747690 16447 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.1
I20260811 19:00:08.753239 16447 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:42795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.1.keytab
--principal=kudu/127.13.109.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.109.1
--webserver_port=0
--tserver_master_addrs=127.13.109.62:43929
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16447
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.1
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:00:08.754516 16447 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:00:08.755563 16447 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:00:08.765664 16452 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:08.765686 16455 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:08.765666 16453 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:08.766041 16447 server_base.cc:1034] running on GCE node
I20260811 19:00:08.766593 16447 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:00:08.767283 16447 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:00:08.768489 16447 hybrid_clock.cc:648] HybridClock initialized: now 1786474808768473 us; error 33 us; skew 500 ppm
Aug 11 19:00:08 dist-test-slave-0kls krb5kdc[16330](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474808, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:00:08.771456 16447 init.cc:377] Logged in from keytab as kudu/127.13.109.1@KRBTEST.COM (short username kudu)
I20260811 19:00:08.772394 16447 webserver.cc:460] Webserver started at http://127.13.109.1:42331/ using document root <none> and password file <none>
I20260811 19:00:08.772919 16447 fs_manager.cc:359] Metadata directory not provided
I20260811 19:00:08.772979 16447 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:00:08.773160 16447 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:00:08.775358 16447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-0/data/instance:
uuid: "278a342f59674eeba2c88fe23d630a44"
format_stamp: "Formatted at 2026-08-11 19:00:08 on dist-test-slave-0kls"
server_key: "13d140bd4feb3b497f6b058b4b24696c"
server_key_iv: "696237f5f88545fdee0db84ca4e88190"
server_key_version: "encryptionkey@0"
I20260811 19:00:08.775872 16447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-0/wal/instance:
uuid: "278a342f59674eeba2c88fe23d630a44"
format_stamp: "Formatted at 2026-08-11 19:00:08 on dist-test-slave-0kls"
server_key: "13d140bd4feb3b497f6b058b4b24696c"
server_key_iv: "696237f5f88545fdee0db84ca4e88190"
server_key_version: "encryptionkey@0"
I20260811 19:00:08.779649 16447 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 19:00:08.782145 16462 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:08.783388 16447 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 19:00:08.783514 16447 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-0/wal
uuid: "278a342f59674eeba2c88fe23d630a44"
format_stamp: "Formatted at 2026-08-11 19:00:08 on dist-test-slave-0kls"
server_key: "13d140bd4feb3b497f6b058b4b24696c"
server_key_iv: "696237f5f88545fdee0db84ca4e88190"
server_key_version: "encryptionkey@0"
I20260811 19:00:08.783627 16447 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:00:08.796382 16447 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:00:08.799908 16447 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:00:08.800272 16447 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:00:08.801014 16447 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:00:08.802109 16447 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:00:08.802165 16447 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:08.802234 16447 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:00:08.802265 16447 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:08.811981 16447 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.1:42863
I20260811 19:00:08.812006 16589 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.1:42863 every 8 connection(s)
I20260811 19:00:08.813097 16447 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
I20260811 19:00:08.815235 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 16447
I20260811 19:00:08.815371 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-0/wal/instance
I20260811 19:00:08.815673 13748 external_mini_cluster.cc:1428] Setting key 39fb6a9765c1116355412fa1610e4346
Aug 11 19:00:08 dist-test-slave-0kls krb5kdc[16330](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474808, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.1@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.109.2@KRBTEST.COM; defaulting to no policy
I20260811 19:00:08.829191 16421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:08.815154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.1:56461 (local address 127.13.109.62:43929)
0811 19:00:08.815346 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:08.815354 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:08.816544 (+ 1190us) server_negotiation.cc:407] Connection header received
0811 19:00:08.817510 (+ 966us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:08.817514 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:08.817565 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:08.817648 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:08.818956 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:08.819858 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:08.821535 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:08.821849 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:08.821956 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:08.824991 (+ 3035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:08.824997 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:08.825003 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:08.825041 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:08.826862 (+ 1821us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:08.827472 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:08.827478 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:08.827482 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:08.827587 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:08.827998 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:08.828001 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:08.828002 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:08.828220 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:08.828303 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:08.828899 (+ 596us) server_negotiation.cc:299] Negotiation successful
0811 19:00:08.829054 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.109.2@KRBTEST.COM" created.
I20260811 19:00:08.830178 16594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:08.815593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:43929 (local address 127.13.109.1:56461)
0811 19:00:08.816388 (+ 795us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:08.816444 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:08.817294 (+ 850us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:08.817918 (+ 624us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:08.817926 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:08.818292 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:08.818798 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:08.818812 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:08.820043 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:08.820046 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:08.821377 (+ 1331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:08.821393 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:08.822036 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:08.822043 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:08.822319 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:08.823056 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:08.823081 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:08.824768 (+ 1687us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:00:08.827017 (+ 2249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:08.827022 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:08.827042 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:08.827343 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:08.827711 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:08.827714 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:08.827715 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:08.827895 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:08.828331 (+ 436us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:00:08.828337 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:00:08.828624 (+ 287us) client_negotiation.cc:769] Sending connection context
0811 19:00:08.828799 (+ 175us) client_negotiation.cc:240] Negotiation successful
0811 19:00:08.829093 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":578,"thread_start_us":185,"threads_started":1}
I20260811 19:00:08.831562 16590 heartbeater.cc:347] Connected to a master server at 127.13.109.62:43929
I20260811 19:00:08.831831 16590 heartbeater.cc:464] Registering TS with master...
I20260811 19:00:08.832408 16590 heartbeater.cc:511] Master 127.13.109.62:43929 requested a full tablet report, sending...
I20260811 19:00:08.834122 16379 ts_manager.cc:194] Registered new tserver with Master: 278a342f59674eeba2c88fe23d630a44 (127.13.109.1:42863)
I20260811 19:00:08.835403 16379 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.109.1@KRBTEST.COM'} at 127.13.109.1:56461
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.109.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.2.keytab.
Entry for principal kudu/127.13.109.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.2.keytab.
I20260811 19:00:08.849623 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.2:0
--local_ip_for_outbound_sockets=127.13.109.2
--webserver_interface=127.13.109.2
--webserver_port=0
--tserver_master_addrs=127.13.109.62:43929
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.2.keytab
--principal=kudu/127.13.109.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:42795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:00:08.970983 16598 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:00:08.971330 16598 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:00:08.971417 16598 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:00:08.975693 16598 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:00:08.975796 16598 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:00:08.975890 16598 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.2
I20260811 19:00:08.981669 16598 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:42795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.2.keytab
--principal=kudu/127.13.109.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.109.2
--webserver_port=0
--tserver_master_addrs=127.13.109.62:43929
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16598
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.2
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:00:08.982976 16598 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:00:08.984125 16598 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:00:08.994930 16606 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:08.994947 16604 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:08.994947 16603 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:08.995217 16598 server_base.cc:1034] running on GCE node
I20260811 19:00:08.995793 16598 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:00:08.996544 16598 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:00:08.997721 16598 hybrid_clock.cc:648] HybridClock initialized: now 1786474808997699 us; error 44 us; skew 500 ppm
Aug 11 19:00:08 dist-test-slave-0kls krb5kdc[16330](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474808, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:00:09.000838 16598 init.cc:377] Logged in from keytab as kudu/127.13.109.2@KRBTEST.COM (short username kudu)
I20260811 19:00:09.002095 16598 webserver.cc:460] Webserver started at http://127.13.109.2:34777/ using document root <none> and password file <none>
I20260811 19:00:09.002767 16598 fs_manager.cc:359] Metadata directory not provided
I20260811 19:00:09.002844 16598 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:00:09.003111 16598 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:00:09.005569 16598 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-1/data/instance:
uuid: "232b5d4f09254e52a9165ac41c5fd786"
format_stamp: "Formatted at 2026-08-11 19:00:09 on dist-test-slave-0kls"
server_key: "21932ce9d72da8d7bd32ecc17ff319b5"
server_key_iv: "c1fc09b6d419fa041751573a5e509a75"
server_key_version: "encryptionkey@0"
I20260811 19:00:09.006323 16598 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-1/wal/instance:
uuid: "232b5d4f09254e52a9165ac41c5fd786"
format_stamp: "Formatted at 2026-08-11 19:00:09 on dist-test-slave-0kls"
server_key: "21932ce9d72da8d7bd32ecc17ff319b5"
server_key_iv: "c1fc09b6d419fa041751573a5e509a75"
server_key_version: "encryptionkey@0"
I20260811 19:00:09.010696 16598 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 19:00:09.013583 16615 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:09.015007 16598 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 19:00:09.015162 16598 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-1/wal
uuid: "232b5d4f09254e52a9165ac41c5fd786"
format_stamp: "Formatted at 2026-08-11 19:00:09 on dist-test-slave-0kls"
server_key: "21932ce9d72da8d7bd32ecc17ff319b5"
server_key_iv: "c1fc09b6d419fa041751573a5e509a75"
server_key_version: "encryptionkey@0"
I20260811 19:00:09.015291 16598 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:00:09.063699 16598 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:00:09.067569 16598 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:00:09.067849 16598 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:00:09.068615 16598 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:00:09.069646 16598 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:00:09.069705 16598 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:09.069756 16598 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:00:09.069859 16598 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260811 19:00:09.080426 16598 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.2:33259
I20260811 19:00:09.080456 16744 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.2:33259 every 8 connection(s)
I20260811 19:00:09.081595 16598 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
I20260811 19:00:09.088683 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 16598
I20260811 19:00:09.088820 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-1/wal/instance
I20260811 19:00:09.089167 13748 external_mini_cluster.cc:1428] Setting key 0bb906c3fd0782fd9718c6eb55d9339f
Aug 11 19:00:09 dist-test-slave-0kls krb5kdc[16330](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474808, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.2@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
I20260811 19:00:09.098656 16421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:09.083808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.2:57283 (local address 127.13.109.62:43929)
0811 19:00:09.083978 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:09.083983 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:09.085336 (+ 1353us) server_negotiation.cc:407] Connection header received
0811 19:00:09.086468 (+ 1132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:09.086471 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:09.086535 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:09.086618 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:09.088174 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:09.088976 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:09.090592 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:09.090896 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:09.091013 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:09.094283 (+ 3270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:09.094287 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:09.094289 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:09.094318 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:09.096024 (+ 1706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:09.096727 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:09.096731 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:09.096732 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:09.096823 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:09.097248 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:09.097252 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:09.097253 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:09.097480 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:09.097605 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:09.098302 (+ 697us) server_negotiation.cc:299] Negotiation successful
0811 19:00:09.098458 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 19:00:09.099609 16747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:09.084284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:43929 (local address 127.13.109.2:57283)
0811 19:00:09.085132 (+ 848us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:09.085182 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:09.086198 (+ 1016us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:09.086780 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:09.086788 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:09.087329 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:09.087928 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:09.087945 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:09.089241 (+ 1296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:09.089244 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:09.090455 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:09.090477 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:09.091114 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:09.091117 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:09.091299 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:09.092025 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:09.092045 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:09.094032 (+ 1987us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:00:09.096204 (+ 2172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:09.096212 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:09.096235 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:09.096579 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:09.096930 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:09.096934 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:09.096936 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:09.097132 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:09.097605 (+ 473us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:00:09.097617 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:00:09.098013 (+ 396us) client_negotiation.cc:769] Sending connection context
0811 19:00:09.098234 (+ 221us) client_negotiation.cc:240] Negotiation successful
0811 19:00:09.098638 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":609,"thread_start_us":125,"threads_started":1}
WARNING: no policy specified for kudu/127.13.109.3@KRBTEST.COM; defaulting to no policy
I20260811 19:00:09.101027 16745 heartbeater.cc:347] Connected to a master server at 127.13.109.62:43929
I20260811 19:00:09.101336 16745 heartbeater.cc:464] Registering TS with master...
I20260811 19:00:09.101997 16745 heartbeater.cc:511] Master 127.13.109.62:43929 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.109.3@KRBTEST.COM" created.
I20260811 19:00:09.103488 16379 ts_manager.cc:194] Registered new tserver with Master: 232b5d4f09254e52a9165ac41c5fd786 (127.13.109.2:33259)
I20260811 19:00:09.104197 16379 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.109.2@KRBTEST.COM'} at 127.13.109.2:57283
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.109.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.3.keytab.
Entry for principal kudu/127.13.109.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.3.keytab.
I20260811 19:00:09.120636 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.3:0
--local_ip_for_outbound_sockets=127.13.109.3
--webserver_interface=127.13.109.3
--webserver_port=0
--tserver_master_addrs=127.13.109.62:43929
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.3.keytab
--principal=kudu/127.13.109.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:42795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:00:09.240157 16750 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:00:09.240482 16750 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:00:09.240530 16750 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:00:09.244779 16750 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:00:09.244871 16750 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:00:09.244935 16750 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.3
I20260811 19:00:09.250783 16750 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:42795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.3.keytab
--principal=kudu/127.13.109.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.109.3
--webserver_port=0
--tserver_master_addrs=127.13.109.62:43929
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16750
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.3
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:00:09.252133 16750 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:00:09.253311 16750 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:00:09.263852 16759 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:09.263866 16757 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:09.263907 16756 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:09.263936 16750 server_base.cc:1034] running on GCE node
I20260811 19:00:09.264611 16750 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:00:09.265381 16750 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:00:09.266592 16750 hybrid_clock.cc:648] HybridClock initialized: now 1786474809266574 us; error 38 us; skew 500 ppm
Aug 11 19:00:09 dist-test-slave-0kls krb5kdc[16330](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474809, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:00:09.269843 16750 init.cc:377] Logged in from keytab as kudu/127.13.109.3@KRBTEST.COM (short username kudu)
I20260811 19:00:09.270840 16750 webserver.cc:460] Webserver started at http://127.13.109.3:43539/ using document root <none> and password file <none>
I20260811 19:00:09.271441 16750 fs_manager.cc:359] Metadata directory not provided
I20260811 19:00:09.271507 16750 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:00:09.271696 16750 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:00:09.274040 16750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-2/data/instance:
uuid: "6fe37f72e287459f9d78eb419add74bb"
format_stamp: "Formatted at 2026-08-11 19:00:09 on dist-test-slave-0kls"
server_key: "fdec825425242020b25d84301d8f76f2"
server_key_iv: "6ac4b6ceed52383279fbcde7e23dc235"
server_key_version: "encryptionkey@0"
I20260811 19:00:09.274713 16750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-2/wal/instance:
uuid: "6fe37f72e287459f9d78eb419add74bb"
format_stamp: "Formatted at 2026-08-11 19:00:09 on dist-test-slave-0kls"
server_key: "fdec825425242020b25d84301d8f76f2"
server_key_iv: "6ac4b6ceed52383279fbcde7e23dc235"
server_key_version: "encryptionkey@0"
I20260811 19:00:09.279155 16750 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 19:00:09.281940 16767 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:09.283311 16750 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 19:00:09.283421 16750 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-2/wal
uuid: "6fe37f72e287459f9d78eb419add74bb"
format_stamp: "Formatted at 2026-08-11 19:00:09 on dist-test-slave-0kls"
server_key: "fdec825425242020b25d84301d8f76f2"
server_key_iv: "6ac4b6ceed52383279fbcde7e23dc235"
server_key_version: "encryptionkey@0"
I20260811 19:00:09.283520 16750 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:00:09.310292 16750 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:00:09.314268 16750 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:00:09.314487 16750 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:00:09.315150 16750 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:00:09.316082 16750 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:00:09.316135 16750 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:09.316182 16750 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:00:09.316200 16750 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:09.326333 16750 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.3:35343
I20260811 19:00:09.326359 16891 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.3:35343 every 8 connection(s)
I20260811 19:00:09.327440 16750 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
I20260811 19:00:09.329054 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 16750
I20260811 19:00:09.329186 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-2/wal/instance
I20260811 19:00:09.329602 13748 external_mini_cluster.cc:1428] Setting key d7c6a87e0f0e0a0a9877ae1a37a55cd8
Aug 11 19:00:09 dist-test-slave-0kls krb5kdc[16330](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474809, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.3@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
I20260811 19:00:09.343369 16421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:09.329708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.3:35931 (local address 127.13.109.62:43929)
0811 19:00:09.329896 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:09.329899 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:09.331502 (+ 1603us) server_negotiation.cc:407] Connection header received
0811 19:00:09.332588 (+ 1086us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:09.332590 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:09.332640 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:09.332714 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:09.333938 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:09.334546 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:09.335698 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:09.336016 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:09.336114 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:09.339353 (+ 3239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:09.339356 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:09.339358 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:09.339384 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:09.340891 (+ 1507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:09.341430 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:09.341433 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:09.341434 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:09.341514 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:09.342049 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:09.342051 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:09.342053 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:09.342239 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:09.342317 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:09.343050 (+ 733us) server_negotiation.cc:299] Negotiation successful
0811 19:00:09.343191 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260811 19:00:09.344548 16894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:09.330228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:43929 (local address 127.13.109.3:35931)
0811 19:00:09.331328 (+ 1100us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:09.331373 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:09.332382 (+ 1009us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:09.332875 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:09.332882 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:09.333267 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:09.333769 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:09.333802 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:09.334671 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:09.334674 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:09.335566 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:09.335576 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:09.336160 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:09.336163 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:09.336410 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:09.337285 (+ 875us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:09.337316 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:09.339164 (+ 1848us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:00:09.341010 (+ 1846us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:09.341015 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:09.341030 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:09.341328 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:09.341640 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:09.341643 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:09.341645 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:09.341909 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:09.342365 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:00:09.342373 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:00:09.342719 (+ 346us) client_negotiation.cc:769] Sending connection context
0811 19:00:09.342944 (+ 225us) client_negotiation.cc:240] Negotiation successful
0811 19:00:09.343358 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":862,"thread_start_us":174,"threads_started":1}
I20260811 19:00:09.345747 16892 heartbeater.cc:347] Connected to a master server at 127.13.109.62:43929
I20260811 19:00:09.346055 16892 heartbeater.cc:464] Registering TS with master...
I20260811 19:00:09.346619 16892 heartbeater.cc:511] Master 127.13.109.62:43929 requested a full tablet report, sending...
I20260811 19:00:09.347776 16379 ts_manager.cc:194] Registered new tserver with Master: 6fe37f72e287459f9d78eb419add74bb (127.13.109.3:35343)
I20260811 19:00:09.348450 16379 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.109.3@KRBTEST.COM'} at 127.13.109.3:35931
I20260811 19:00:09.356468 13748 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 19:00:09.373731 16421 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:00:09.371918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39804 (local address 127.13.109.62:43929)
0811 19:00:09.372066 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:09.372070 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:09.372270 (+ 200us) server_negotiation.cc:407] Connection header received
0811 19:00:09.372411 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:09.372416 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:09.372481 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:09.372591 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:09.373598 (+ 1007us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39804: BlockingRecv error: recv got EOF from 127.0.0.1:39804 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 19:00:09.375029 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 16447
I20260811 19:00:09.386540 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 16598
I20260811 19:00:09.399308 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 16750
I20260811 19:00:09.408342 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 16344
2026-08-11T19:00:09Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2344 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474792496492-13748-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:00:09 dist-test-slave-0kls krb5kdc[16904](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:00:09 dist-test-slave-0kls krb5kdc[16904](info): set up 2 sockets
Aug 11 19:00:09 dist-test-slave-0kls krb5kdc[16904](info): commencing operation
krb5kdc: starting...
W20260811 19:00:10.502246 13748 mini_kdc.cc:121] Time spent starting KDC: real 1.069s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474792496492-13748-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474792496492-13748-0/krb5kdc/test-user.keytab.
Aug 11 19:00:10 dist-test-slave-0kls krb5kdc[16904](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474810, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:00:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:00:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.109.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.109.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.109.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab.
Entry for principal kudu/127.13.109.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab.
I20260811 19:00:10.637442 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474792496492-13748-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474792496492-13748-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474792496492-13748-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.62:38165
--webserver_interface=127.13.109.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab
--principal=kudu/127.13.109.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:42561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.109.62:38165
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:00:10.758808 16921 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:00:10.759166 16921 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:00:10.759255 16921 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:00:10.763475 16921 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:00:10.763602 16921 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:00:10.763643 16921 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:00:10.763680 16921 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:00:10.763720 16921 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:00:10.769439 16921 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:42561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474792496492-13748-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474792496492-13748-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.109.62:38165
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab
--principal=kudu/127.13.109.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.62:38165
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.109.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16921
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474792496492-13748-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:00:10.770779 16921 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:00:10.771942 16921 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:00:10.781301 16931 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:10.781430 16921 server_base.cc:1034] running on GCE node
W20260811 19:00:10.781275 16929 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:10.781262 16934 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:10.782250 16921 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:00:10.783509 16921 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:00:10.784735 16921 hybrid_clock.cc:648] HybridClock initialized: now 1786474810784721 us; error 29 us; skew 500 ppm
Aug 11 19:00:10 dist-test-slave-0kls krb5kdc[16904](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474810, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:00:10.801648 16921 init.cc:377] Logged in from keytab as kudu/127.13.109.62@KRBTEST.COM (short username kudu)
I20260811 19:00:10.802850 16921 webserver.cc:460] Webserver started at http://127.13.109.62:42975/ using document root <none> and password file <none>
I20260811 19:00:10.803516 16921 fs_manager.cc:359] Metadata directory not provided
I20260811 19:00:10.803633 16921 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:00:10.803905 16921 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:00:10.806303 16921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474792496492-13748-0/minicluster-data/master-0/data/instance:
uuid: "3ce878729ff64db895be1460825ec66c"
format_stamp: "Formatted at 2026-08-11 19:00:10 on dist-test-slave-0kls"
server_key: "27e2aecafdb907afb12573c16981677b"
server_key_iv: "94ad5599f755758c2e3e1daf6a512148"
server_key_version: "encryptionkey@0"
I20260811 19:00:10.807116 16921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474792496492-13748-0/minicluster-data/master-0/wal/instance:
uuid: "3ce878729ff64db895be1460825ec66c"
format_stamp: "Formatted at 2026-08-11 19:00:10 on dist-test-slave-0kls"
server_key: "27e2aecafdb907afb12573c16981677b"
server_key_iv: "94ad5599f755758c2e3e1daf6a512148"
server_key_version: "encryptionkey@0"
I20260811 19:00:10.811394 16921 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 19:00:10.814147 16941 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:10.815632 16921 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 19:00:10.815796 16921 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474792496492-13748-0/minicluster-data/master-0/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474792496492-13748-0/minicluster-data/master-0/wal
uuid: "3ce878729ff64db895be1460825ec66c"
format_stamp: "Formatted at 2026-08-11 19:00:10 on dist-test-slave-0kls"
server_key: "27e2aecafdb907afb12573c16981677b"
server_key_iv: "94ad5599f755758c2e3e1daf6a512148"
server_key_version: "encryptionkey@0"
I20260811 19:00:10.815932 16921 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474792496492-13748-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474792496492-13748-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474792496492-13748-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:00:10.836097 16921 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:00:10.839422 16921 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:00:10.839666 16921 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:00:10.847616 16921 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.62:38165
I20260811 19:00:10.847926 16993 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.62:38165 every 8 connection(s)
I20260811 19:00:10.848747 16921 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
I20260811 19:00:10.852036 16994 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:00:10.855986 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 16921
I20260811 19:00:10.856146 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474792496492-13748-0/minicluster-data/master-0/wal/instance
I20260811 19:00:10.856431 13748 external_mini_cluster.cc:1428] Setting key 0dc884e0d7932d859b0f59eb43ab4d51
I20260811 19:00:10.858474 16994 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ce878729ff64db895be1460825ec66c: Bootstrap starting.
I20260811 19:00:10.861372 16994 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3ce878729ff64db895be1460825ec66c: Neither blocks nor log segments found. Creating new log.
I20260811 19:00:10.862329 16994 log.cc:824] T 00000000000000000000000000000000 P 3ce878729ff64db895be1460825ec66c: Log is configured to *not* fsync() on all Append() calls
I20260811 19:00:10.864801 16994 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ce878729ff64db895be1460825ec66c: No bootstrap required, opened a new log
Aug 11 19:00:10 dist-test-slave-0kls krb5kdc[16904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474810, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
I20260811 19:00:10.869760 16994 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3ce878729ff64db895be1460825ec66c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ce878729ff64db895be1460825ec66c" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 38165 } }
I20260811 19:00:10.870110 16994 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3ce878729ff64db895be1460825ec66c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:00:10.870196 16994 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3ce878729ff64db895be1460825ec66c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce878729ff64db895be1460825ec66c, State: Initialized, Role: FOLLOWER
I20260811 19:00:10.870781 16994 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3ce878729ff64db895be1460825ec66c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ce878729ff64db895be1460825ec66c" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 38165 } }
I20260811 19:00:10.870997 16994 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3ce878729ff64db895be1460825ec66c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:00:10.871081 16994 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3ce878729ff64db895be1460825ec66c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:00:10.871189 16994 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3ce878729ff64db895be1460825ec66c [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:00:10.872534 16994 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3ce878729ff64db895be1460825ec66c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ce878729ff64db895be1460825ec66c" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 38165 } }
I20260811 19:00:10.872954 16994 leader_election.cc:302] T 00000000000000000000000000000000 P 3ce878729ff64db895be1460825ec66c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3ce878729ff64db895be1460825ec66c; no voters:
I20260811 19:00:10.873333 16994 leader_election.cc:288] T 00000000000000000000000000000000 P 3ce878729ff64db895be1460825ec66c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:00:10.873487 16999 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3ce878729ff64db895be1460825ec66c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:00:10.873565 16997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:10.857966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52224 (local address 127.13.109.62:38165)
0811 19:00:10.858440 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:10.858461 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:10.858508 (+ 47us) server_negotiation.cc:407] Connection header received
0811 19:00:10.859369 (+ 861us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:10.859394 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:10.859863 (+ 469us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:10.860249 (+ 386us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:10.860999 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:10.862557 (+ 1558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:10.863689 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:10.864376 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:10.864550 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:10.867553 (+ 3003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:10.867561 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:10.867584 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:10.867631 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:10.869856 (+ 2225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:10.870459 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:10.870465 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:10.870472 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:10.870636 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:10.870927 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:10.870931 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:10.870932 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:10.871558 (+ 626us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:10.871762 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:10.872056 (+ 294us) server_negotiation.cc:299] Negotiation successful
0811 19:00:10.872432 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":132,"threads_started":1}
I20260811 19:00:10.873754 16999 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3ce878729ff64db895be1460825ec66c [term 1 LEADER]: Becoming Leader. State: Replica: 3ce878729ff64db895be1460825ec66c, State: Running, Role: LEADER
I20260811 19:00:10.874150 16999 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3ce878729ff64db895be1460825ec66c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ce878729ff64db895be1460825ec66c" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 38165 } }
I20260811 19:00:10.874718 16994 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3ce878729ff64db895be1460825ec66c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:00:10.875473 16999 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ce878729ff64db895be1460825ec66c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3ce878729ff64db895be1460825ec66c. Latest consensus state: current_term: 1 leader_uuid: "3ce878729ff64db895be1460825ec66c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ce878729ff64db895be1460825ec66c" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 38165 } } }
I20260811 19:00:10.875474 17001 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ce878729ff64db895be1460825ec66c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3ce878729ff64db895be1460825ec66c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ce878729ff64db895be1460825ec66c" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 38165 } } }
I20260811 19:00:10.875677 17001 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ce878729ff64db895be1460825ec66c [sys.catalog]: This master's current role is: LEADER
I20260811 19:00:10.875677 16999 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ce878729ff64db895be1460825ec66c [sys.catalog]: This master's current role is: LEADER
I20260811 19:00:10.876714 17008 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:00:10.877199 13748 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 19:00:10.880301 17008 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:00:10.887475 17008 catalog_manager.cc:1347] Generated new cluster ID: 748870c427ad4db9b7452e08ac4b7cee
I20260811 19:00:10.887686 17008 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:00:10.899331 17008 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:00:10.900568 17008 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:00:10.902724 16997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:10.896612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52230 (local address 127.13.109.62:38165)
0811 19:00:10.896750 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:10.896755 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:10.896981 (+ 226us) server_negotiation.cc:407] Connection header received
0811 19:00:10.897121 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:10.897124 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:10.897183 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:10.897287 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:10.898044 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:10.898749 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:10.900019 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:10.900379 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:10.900505 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:10.901547 (+ 1042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:10.901551 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:10.901554 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:00:10.902175 (+ 621us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:10.902222 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:00:10.902243 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:10.902332 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:10.902469 (+ 137us) server_negotiation.cc:299] Negotiation successful
0811 19:00:10.902561 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":54}
I20260811 19:00:10.908480 17008 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3ce878729ff64db895be1460825ec66c: Generated new TSK 0
I20260811 19:00:10.909528 17008 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 19:00:10.918929 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 16921
2026-08-11T19:00:10Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1508 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:00:10 dist-test-slave-0kls krb5kdc[17034](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:00:10 dist-test-slave-0kls krb5kdc[17034](info): set up 2 sockets
Aug 11 19:00:10 dist-test-slave-0kls krb5kdc[17034](info): commencing operation
krb5kdc: starting...
W20260811 19:00:12.021629 13748 mini_kdc.cc:121] Time spent starting KDC: real 1.079s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/krb5kdc/test-user.keytab.
Aug 11 19:00:12 dist-test-slave-0kls krb5kdc[17034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474812, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:00:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:00:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.109.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.109.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.109.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab.
Entry for principal kudu/127.13.109.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab.
I20260811 19:00:12.165925 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.62:40817
--webserver_interface=127.13.109.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab
--principal=kudu/127.13.109.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:40651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.109.62:40817
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:00:12.284503 17048 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:00:12.284890 17048 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:00:12.284976 17048 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:00:12.289302 17048 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:00:12.289429 17048 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:00:12.289469 17048 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:00:12.289510 17048 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:00:12.289541 17048 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:00:12.295303 17048 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:40651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.109.62:40817
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab
--principal=kudu/127.13.109.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.62:40817
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.109.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17048
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:00:12.296545 17048 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:00:12.297643 17048 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:00:12.306600 17053 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:12.306582 17054 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:12.306583 17056 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:12.306643 17048 server_base.cc:1034] running on GCE node
I20260811 19:00:12.307451 17048 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:00:12.308681 17048 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:00:12.309872 17048 hybrid_clock.cc:648] HybridClock initialized: now 1786474812309828 us; error 57 us; skew 500 ppm
Aug 11 19:00:12 dist-test-slave-0kls krb5kdc[17034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474812, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:00:12.312713 17048 init.cc:377] Logged in from keytab as kudu/127.13.109.62@KRBTEST.COM (short username kudu)
I20260811 19:00:12.313735 17048 webserver.cc:460] Webserver started at http://127.13.109.62:33063/ using document root <none> and password file <none>
I20260811 19:00:12.314364 17048 fs_manager.cc:359] Metadata directory not provided
I20260811 19:00:12.314463 17048 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:00:12.314718 17048 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:00:12.316876 17048 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/minicluster-data/master-0/data/instance:
uuid: "ff32b4c2dd0c427ba4df812efde7623a"
format_stamp: "Formatted at 2026-08-11 19:00:12 on dist-test-slave-0kls"
server_key: "d7cf47809c6678d7fa06ec98433e17d9"
server_key_iv: "e9111414099175dd556020231b8f32b1"
server_key_version: "encryptionkey@0"
I20260811 19:00:12.317482 17048 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/minicluster-data/master-0/wal/instance:
uuid: "ff32b4c2dd0c427ba4df812efde7623a"
format_stamp: "Formatted at 2026-08-11 19:00:12 on dist-test-slave-0kls"
server_key: "d7cf47809c6678d7fa06ec98433e17d9"
server_key_iv: "e9111414099175dd556020231b8f32b1"
server_key_version: "encryptionkey@0"
I20260811 19:00:12.321590 17048 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260811 19:00:12.324448 17065 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:12.325910 17048 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 19:00:12.326056 17048 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/minicluster-data/master-0/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/minicluster-data/master-0/wal
uuid: "ff32b4c2dd0c427ba4df812efde7623a"
format_stamp: "Formatted at 2026-08-11 19:00:12 on dist-test-slave-0kls"
server_key: "d7cf47809c6678d7fa06ec98433e17d9"
server_key_iv: "e9111414099175dd556020231b8f32b1"
server_key_version: "encryptionkey@0"
I20260811 19:00:12.326184 17048 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:00:12.351167 17048 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:00:12.354324 17048 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:00:12.354544 17048 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:00:12.361527 17048 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.62:40817
I20260811 19:00:12.361879 17119 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.62:40817 every 8 connection(s)
I20260811 19:00:12.362607 17048 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
I20260811 19:00:12.364204 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 17048
I20260811 19:00:12.364311 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/minicluster-data/master-0/wal/instance
I20260811 19:00:12.364624 13748 external_mini_cluster.cc:1428] Setting key fde56daab64c52fdd02cc6b269143df3
I20260811 19:00:12.366060 17120 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:00:12.373589 17120 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff32b4c2dd0c427ba4df812efde7623a: Bootstrap starting.
Aug 11 19:00:12 dist-test-slave-0kls krb5kdc[17034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474812, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
I20260811 19:00:12.376426 17120 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ff32b4c2dd0c427ba4df812efde7623a: Neither blocks nor log segments found. Creating new log.
I20260811 19:00:12.377406 17120 log.cc:824] T 00000000000000000000000000000000 P ff32b4c2dd0c427ba4df812efde7623a: Log is configured to *not* fsync() on all Append() calls
I20260811 19:00:12.380216 17120 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff32b4c2dd0c427ba4df812efde7623a: No bootstrap required, opened a new log
I20260811 19:00:12.380975 17123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:12.366292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46360 (local address 127.13.109.62:40817)
0811 19:00:12.366749 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:12.366769 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:12.366815 (+ 46us) server_negotiation.cc:407] Connection header received
0811 19:00:12.367708 (+ 893us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:12.367735 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:12.368193 (+ 458us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:12.368596 (+ 403us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:12.369350 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:12.370922 (+ 1572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:12.371812 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:12.372551 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:12.372715 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:12.375340 (+ 2625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:12.375347 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:12.375364 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:12.375402 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:12.377364 (+ 1962us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:12.377866 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:12.377873 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:12.377880 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:12.378013 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:12.378422 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:12.378426 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:12.378428 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:12.378993 (+ 565us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:12.379277 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:12.379622 (+ 345us) server_negotiation.cc:299] Negotiation successful
0811 19:00:12.379964 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":300,"thread_start_us":127,"threads_started":1}
I20260811 19:00:12.384498 17120 raft_consensus.cc:348] T 00000000000000000000000000000000 P ff32b4c2dd0c427ba4df812efde7623a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff32b4c2dd0c427ba4df812efde7623a" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 40817 } }
I20260811 19:00:12.384756 17120 raft_consensus.cc:374] T 00000000000000000000000000000000 P ff32b4c2dd0c427ba4df812efde7623a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:00:12.384792 17120 raft_consensus.cc:729] T 00000000000000000000000000000000 P ff32b4c2dd0c427ba4df812efde7623a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff32b4c2dd0c427ba4df812efde7623a, State: Initialized, Role: FOLLOWER
I20260811 19:00:12.385233 17120 consensus_queue.cc:260] T 00000000000000000000000000000000 P ff32b4c2dd0c427ba4df812efde7623a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff32b4c2dd0c427ba4df812efde7623a" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 40817 } }
I20260811 19:00:12.385361 17120 raft_consensus.cc:388] T 00000000000000000000000000000000 P ff32b4c2dd0c427ba4df812efde7623a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:00:12.385411 17120 raft_consensus.cc:482] T 00000000000000000000000000000000 P ff32b4c2dd0c427ba4df812efde7623a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:00:12.385483 17120 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ff32b4c2dd0c427ba4df812efde7623a [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:00:12.386674 17120 raft_consensus.cc:504] T 00000000000000000000000000000000 P ff32b4c2dd0c427ba4df812efde7623a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff32b4c2dd0c427ba4df812efde7623a" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 40817 } }
I20260811 19:00:12.387029 17120 leader_election.cc:302] T 00000000000000000000000000000000 P ff32b4c2dd0c427ba4df812efde7623a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ff32b4c2dd0c427ba4df812efde7623a; no voters:
I20260811 19:00:12.387342 17120 leader_election.cc:288] T 00000000000000000000000000000000 P ff32b4c2dd0c427ba4df812efde7623a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:00:12.387480 17125 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ff32b4c2dd0c427ba4df812efde7623a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:00:12.387782 17125 raft_consensus.cc:686] T 00000000000000000000000000000000 P ff32b4c2dd0c427ba4df812efde7623a [term 1 LEADER]: Becoming Leader. State: Replica: ff32b4c2dd0c427ba4df812efde7623a, State: Running, Role: LEADER
I20260811 19:00:12.388195 17125 consensus_queue.cc:237] T 00000000000000000000000000000000 P ff32b4c2dd0c427ba4df812efde7623a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff32b4c2dd0c427ba4df812efde7623a" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 40817 } }
I20260811 19:00:12.388731 17120 sys_catalog.cc:564] T 00000000000000000000000000000000 P ff32b4c2dd0c427ba4df812efde7623a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:00:12.390051 17127 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff32b4c2dd0c427ba4df812efde7623a [sys.catalog]: SysCatalogTable state changed. Reason: New leader ff32b4c2dd0c427ba4df812efde7623a. Latest consensus state: current_term: 1 leader_uuid: "ff32b4c2dd0c427ba4df812efde7623a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff32b4c2dd0c427ba4df812efde7623a" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 40817 } } }
I20260811 19:00:12.390199 17127 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff32b4c2dd0c427ba4df812efde7623a [sys.catalog]: This master's current role is: LEADER
I20260811 19:00:12.390522 17126 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff32b4c2dd0c427ba4df812efde7623a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ff32b4c2dd0c427ba4df812efde7623a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff32b4c2dd0c427ba4df812efde7623a" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 40817 } } }
I20260811 19:00:12.390616 17135 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:00:12.390709 17126 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff32b4c2dd0c427ba4df812efde7623a [sys.catalog]: This master's current role is: LEADER
I20260811 19:00:12.393558 17135 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:00:12.399315 17135 catalog_manager.cc:1347] Generated new cluster ID: 12fee189ff32473daf9d9e382f540338
I20260811 19:00:12.399441 17135 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:00:12.408317 17135 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:00:12.409319 17135 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:00:12.420493 17135 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ff32b4c2dd0c427ba4df812efde7623a: Generated new TSK 0
I20260811 19:00:12.421298 17135 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.109.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.109.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.109.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.1.keytab.
Entry for principal kudu/127.13.109.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.1.keytab.
I20260811 19:00:12.457755 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.1:0
--local_ip_for_outbound_sockets=127.13.109.1
--webserver_interface=127.13.109.1
--webserver_port=0
--tserver_master_addrs=127.13.109.62:40817
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.1.keytab
--principal=kudu/127.13.109.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:40651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:00:12.583242 17149 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:00:12.583549 17149 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:00:12.583597 17149 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:00:12.587854 17149 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:00:12.587927 17149 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:00:12.587990 17149 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.1
I20260811 19:00:12.593745 17149 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:40651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.1.keytab
--principal=kudu/127.13.109.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.109.1
--webserver_port=0
--tserver_master_addrs=127.13.109.62:40817
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17149
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.1
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:00:12.595094 17149 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:00:12.596177 17149 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:00:12.606767 17157 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:12.606781 17154 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:12.607002 17149 server_base.cc:1034] running on GCE node
W20260811 19:00:12.607082 17155 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:12.607566 17149 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:00:12.608295 17149 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:00:12.609508 17149 hybrid_clock.cc:648] HybridClock initialized: now 1786474812609468 us; error 54 us; skew 500 ppm
Aug 11 19:00:12 dist-test-slave-0kls krb5kdc[17034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474812, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:00:12.612739 17149 init.cc:377] Logged in from keytab as kudu/127.13.109.1@KRBTEST.COM (short username kudu)
I20260811 19:00:12.613873 17149 webserver.cc:460] Webserver started at http://127.13.109.1:45935/ using document root <none> and password file <none>
I20260811 19:00:12.614526 17149 fs_manager.cc:359] Metadata directory not provided
I20260811 19:00:12.614635 17149 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:00:12.614897 17149 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:00:12.617692 17149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-0/data/instance:
uuid: "888095c84e6a4b1b803c9b1c1290d959"
format_stamp: "Formatted at 2026-08-11 19:00:12 on dist-test-slave-0kls"
server_key: "500309ac6c05bf25813272e5aa08d842"
server_key_iv: "24702b447b0b2642614d5a789c07b69c"
server_key_version: "encryptionkey@0"
I20260811 19:00:12.618444 17149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-0/wal/instance:
uuid: "888095c84e6a4b1b803c9b1c1290d959"
format_stamp: "Formatted at 2026-08-11 19:00:12 on dist-test-slave-0kls"
server_key: "500309ac6c05bf25813272e5aa08d842"
server_key_iv: "24702b447b0b2642614d5a789c07b69c"
server_key_version: "encryptionkey@0"
I20260811 19:00:12.622787 17149 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 19:00:12.625501 17165 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:12.627069 17149 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 19:00:12.627239 17149 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-0/wal
uuid: "888095c84e6a4b1b803c9b1c1290d959"
format_stamp: "Formatted at 2026-08-11 19:00:12 on dist-test-slave-0kls"
server_key: "500309ac6c05bf25813272e5aa08d842"
server_key_iv: "24702b447b0b2642614d5a789c07b69c"
server_key_version: "encryptionkey@0"
I20260811 19:00:12.627383 17149 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:00:12.648334 17149 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:00:12.652061 17149 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:00:12.652254 17149 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:00:12.652863 17149 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:00:12.653772 17149 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:00:12.653904 17149 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:12.653952 17149 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:00:12.653973 17149 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:12.663249 17149 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.1:34443
I20260811 19:00:12.663352 17284 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.1:34443 every 8 connection(s)
I20260811 19:00:12.664299 17149 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
I20260811 19:00:12.666510 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 17149
I20260811 19:00:12.666652 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-0/wal/instance
I20260811 19:00:12.666980 13748 external_mini_cluster.cc:1428] Setting key 7a292386462f950fab1858cf8022f268
Aug 11 19:00:12 dist-test-slave-0kls krb5kdc[17034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474812, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.1@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.109.2@KRBTEST.COM; defaulting to no policy
I20260811 19:00:12.681283 17123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:12.666368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.1:41471 (local address 127.13.109.62:40817)
0811 19:00:12.666522 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:12.666526 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:12.667752 (+ 1226us) server_negotiation.cc:407] Connection header received
0811 19:00:12.669446 (+ 1694us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:12.669450 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:12.669506 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:12.669587 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:12.671655 (+ 2068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:12.672539 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:12.673874 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:12.674122 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:12.674246 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:12.676922 (+ 2676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:12.676925 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:12.676927 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:12.676956 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:12.678543 (+ 1587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:12.679110 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:12.679119 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:12.679120 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:12.679201 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:12.679636 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:12.679640 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:12.679642 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:12.679934 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:12.680056 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:12.680939 (+ 883us) server_negotiation.cc:299] Negotiation successful
0811 19:00:12.681091 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.109.2@KRBTEST.COM" created.
I20260811 19:00:12.682400 17287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:12.666931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:40817 (local address 127.13.109.1:41471)
0811 19:00:12.667558 (+ 627us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:12.667624 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:12.669215 (+ 1591us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:12.669831 (+ 616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:12.669843 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:12.670399 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:12.671084 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:12.671104 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:12.672683 (+ 1579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:12.672686 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:12.673698 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:12.673709 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:12.674331 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:12.674336 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:12.674522 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:12.675189 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:12.675214 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:12.676723 (+ 1509us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:00:12.678673 (+ 1950us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:12.678680 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:12.678700 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:12.678989 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:12.679297 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:12.679300 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:12.679302 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:12.679491 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:12.680092 (+ 601us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:00:12.680100 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:00:12.680569 (+ 469us) client_negotiation.cc:769] Sending connection context
0811 19:00:12.680846 (+ 277us) client_negotiation.cc:240] Negotiation successful
0811 19:00:12.681240 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":165,"threads_started":1}
I20260811 19:00:12.684039 17285 heartbeater.cc:347] Connected to a master server at 127.13.109.62:40817
I20260811 19:00:12.684379 17285 heartbeater.cc:464] Registering TS with master...
I20260811 19:00:12.684996 17285 heartbeater.cc:511] Master 127.13.109.62:40817 requested a full tablet report, sending...
I20260811 19:00:12.686968 17082 ts_manager.cc:194] Registered new tserver with Master: 888095c84e6a4b1b803c9b1c1290d959 (127.13.109.1:34443)
I20260811 19:00:12.688318 17082 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.109.1@KRBTEST.COM'} at 127.13.109.1:41471
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.109.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.2.keytab.
Entry for principal kudu/127.13.109.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.2.keytab.
I20260811 19:00:12.702533 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.2:0
--local_ip_for_outbound_sockets=127.13.109.2
--webserver_interface=127.13.109.2
--webserver_port=0
--tserver_master_addrs=127.13.109.62:40817
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.2.keytab
--principal=kudu/127.13.109.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:40651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:00:12.822857 17290 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:00:12.823154 17290 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:00:12.823201 17290 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:00:12.827414 17290 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:00:12.827487 17290 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:00:12.827548 17290 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.2
I20260811 19:00:12.833546 17290 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:40651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.2.keytab
--principal=kudu/127.13.109.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.109.2
--webserver_port=0
--tserver_master_addrs=127.13.109.62:40817
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17290
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.2
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:00:12.834998 17290 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:00:12.836210 17290 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:00:12.846958 17297 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:12.846993 17298 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:12.847052 17290 server_base.cc:1034] running on GCE node
W20260811 19:00:12.846993 17300 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:12.847584 17290 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:00:12.848272 17290 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:00:12.849437 17290 hybrid_clock.cc:648] HybridClock initialized: now 1786474812849421 us; error 28 us; skew 500 ppm
Aug 11 19:00:12 dist-test-slave-0kls krb5kdc[17034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474812, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:00:12.852257 17290 init.cc:377] Logged in from keytab as kudu/127.13.109.2@KRBTEST.COM (short username kudu)
I20260811 19:00:12.853161 17290 webserver.cc:460] Webserver started at http://127.13.109.2:37819/ using document root <none> and password file <none>
I20260811 19:00:12.853749 17290 fs_manager.cc:359] Metadata directory not provided
I20260811 19:00:12.853897 17290 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:00:12.854156 17290 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:00:12.856245 17290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-1/data/instance:
uuid: "7e66647b990b4c8da2300c5d68b55070"
format_stamp: "Formatted at 2026-08-11 19:00:12 on dist-test-slave-0kls"
server_key: "991a27578530d1f2f790ec84d20a3437"
server_key_iv: "83fd984d8a696fb363377df6c24120d8"
server_key_version: "encryptionkey@0"
I20260811 19:00:12.856827 17290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-1/wal/instance:
uuid: "7e66647b990b4c8da2300c5d68b55070"
format_stamp: "Formatted at 2026-08-11 19:00:12 on dist-test-slave-0kls"
server_key: "991a27578530d1f2f790ec84d20a3437"
server_key_iv: "83fd984d8a696fb363377df6c24120d8"
server_key_version: "encryptionkey@0"
I20260811 19:00:12.860718 17290 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 19:00:12.863265 17307 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:12.864517 17290 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 19:00:12.864694 17290 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-1/wal
uuid: "7e66647b990b4c8da2300c5d68b55070"
format_stamp: "Formatted at 2026-08-11 19:00:12 on dist-test-slave-0kls"
server_key: "991a27578530d1f2f790ec84d20a3437"
server_key_iv: "83fd984d8a696fb363377df6c24120d8"
server_key_version: "encryptionkey@0"
I20260811 19:00:12.864820 17290 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:00:12.934024 17290 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:00:12.937642 17290 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:00:12.937901 17290 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:00:12.938594 17290 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:00:12.939486 17290 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:00:12.939559 17290 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:12.939631 17290 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:00:12.939684 17290 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:12.949491 17290 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.2:41923
I20260811 19:00:12.949517 17426 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.2:41923 every 8 connection(s)
I20260811 19:00:12.950502 17290 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
I20260811 19:00:12.951375 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 17290
I20260811 19:00:12.951512 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-1/wal/instance
I20260811 19:00:12.951797 13748 external_mini_cluster.cc:1428] Setting key b3300d7daf1afbd8ddbac6aef8201e1d
Aug 11 19:00:12 dist-test-slave-0kls krb5kdc[17034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474812, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.2@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.109.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.109.3@KRBTEST.COM" created.
I20260811 19:00:12.965648 17123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:12.952564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.2:54741 (local address 127.13.109.62:40817)
0811 19:00:12.952704 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:12.952708 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:12.953841 (+ 1133us) server_negotiation.cc:407] Connection header received
0811 19:00:12.955256 (+ 1415us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:12.955259 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:12.955316 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:12.955399 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:12.956621 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:12.957158 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:12.958450 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:12.958693 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:12.958827 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:12.961879 (+ 3052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:12.961882 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:12.961885 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:12.961914 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:12.963332 (+ 1418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:12.963920 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:12.963924 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:12.963926 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:12.964011 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:12.964389 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:12.964393 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:12.964395 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:12.964602 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:12.964716 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:12.965327 (+ 611us) server_negotiation.cc:299] Negotiation successful
0811 19:00:12.965478 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 19:00:12.966620 17429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:12.953100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:40817 (local address 127.13.109.2:54741)
0811 19:00:12.953636 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:12.953691 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:12.955033 (+ 1342us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:12.955583 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:12.955591 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:12.955969 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:12.956470 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:12.956484 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:12.957308 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:12.957312 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:12.958313 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:12.958324 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:12.958880 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:12.958884 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:12.959082 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:12.959809 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:12.959828 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:12.961623 (+ 1795us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:00:12.963483 (+ 1860us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:12.963490 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:12.963517 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:12.963808 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:12.964113 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:12.964117 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:12.964119 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:12.964284 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:12.964725 (+ 441us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:00:12.964735 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:00:12.965027 (+ 292us) client_negotiation.cc:769] Sending connection context
0811 19:00:12.965281 (+ 254us) client_negotiation.cc:240] Negotiation successful
0811 19:00:12.965653 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":145,"threads_started":1}
I20260811 19:00:12.968012 17427 heartbeater.cc:347] Connected to a master server at 127.13.109.62:40817
I20260811 19:00:12.968331 17427 heartbeater.cc:464] Registering TS with master...
I20260811 19:00:12.968936 17427 heartbeater.cc:511] Master 127.13.109.62:40817 requested a full tablet report, sending...
I20260811 19:00:12.970186 17082 ts_manager.cc:194] Registered new tserver with Master: 7e66647b990b4c8da2300c5d68b55070 (127.13.109.2:41923)
I20260811 19:00:12.970824 17082 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.109.2@KRBTEST.COM'} at 127.13.109.2:54741
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.109.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.3.keytab.
Entry for principal kudu/127.13.109.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.3.keytab.
I20260811 19:00:12.983228 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.3:0
--local_ip_for_outbound_sockets=127.13.109.3
--webserver_interface=127.13.109.3
--webserver_port=0
--tserver_master_addrs=127.13.109.62:40817
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.3.keytab
--principal=kudu/127.13.109.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:40651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:00:13.102571 17433 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:00:13.102874 17433 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:00:13.102921 17433 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:00:13.107123 17433 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:00:13.107247 17433 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:00:13.107369 17433 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.3
I20260811 19:00:13.113654 17433 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:40651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.3.keytab
--principal=kudu/127.13.109.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.109.3
--webserver_port=0
--tserver_master_addrs=127.13.109.62:40817
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17433
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.3
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:00:13.114974 17433 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:00:13.116143 17433 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:00:13.127396 17439 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:13.127416 17441 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:13.127396 17438 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:13.127529 17433 server_base.cc:1034] running on GCE node
I20260811 19:00:13.128199 17433 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:00:13.129055 17433 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:00:13.130263 17433 hybrid_clock.cc:648] HybridClock initialized: now 1786474813130228 us; error 47 us; skew 500 ppm
Aug 11 19:00:13 dist-test-slave-0kls krb5kdc[17034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474813, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:00:13.133484 17433 init.cc:377] Logged in from keytab as kudu/127.13.109.3@KRBTEST.COM (short username kudu)
I20260811 19:00:13.134544 17433 webserver.cc:460] Webserver started at http://127.13.109.3:37649/ using document root <none> and password file <none>
I20260811 19:00:13.135113 17433 fs_manager.cc:359] Metadata directory not provided
I20260811 19:00:13.135176 17433 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:00:13.135355 17433 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:00:13.137451 17433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-2/data/instance:
uuid: "55ed9fc42c164418bfbbaacbf2c5a200"
format_stamp: "Formatted at 2026-08-11 19:00:13 on dist-test-slave-0kls"
server_key: "23c5d6e9fa81443b642a9b0bec1260ff"
server_key_iv: "be88a65417d87fc7aec1390c15a52636"
server_key_version: "encryptionkey@0"
I20260811 19:00:13.138067 17433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-2/wal/instance:
uuid: "55ed9fc42c164418bfbbaacbf2c5a200"
format_stamp: "Formatted at 2026-08-11 19:00:13 on dist-test-slave-0kls"
server_key: "23c5d6e9fa81443b642a9b0bec1260ff"
server_key_iv: "be88a65417d87fc7aec1390c15a52636"
server_key_version: "encryptionkey@0"
I20260811 19:00:13.141945 17433 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 19:00:13.144433 17448 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:13.145741 17433 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 19:00:13.145916 17433 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-2/wal
uuid: "55ed9fc42c164418bfbbaacbf2c5a200"
format_stamp: "Formatted at 2026-08-11 19:00:13 on dist-test-slave-0kls"
server_key: "23c5d6e9fa81443b642a9b0bec1260ff"
server_key_iv: "be88a65417d87fc7aec1390c15a52636"
server_key_version: "encryptionkey@0"
I20260811 19:00:13.146040 17433 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:00:13.160462 17433 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:00:13.164167 17433 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:00:13.164397 17433 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:00:13.165073 17433 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:00:13.166112 17433 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:00:13.166184 17433 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:13.166255 17433 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:00:13.166301 17433 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:13.175875 17433 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.3:37607
I20260811 19:00:13.175902 17567 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.3:37607 every 8 connection(s)
I20260811 19:00:13.176899 17433 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
I20260811 19:00:13.181344 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 17433
I20260811 19:00:13.181502 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474792496492-13748-0/minicluster-data/ts-2/wal/instance
I20260811 19:00:13.181890 13748 external_mini_cluster.cc:1428] Setting key 09effcc3d0ab6e114e00b121c6384ad5
Aug 11 19:00:13 dist-test-slave-0kls krb5kdc[17034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474813, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.3@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
I20260811 19:00:13.191972 17123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:13.178975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.3:39811 (local address 127.13.109.62:40817)
0811 19:00:13.179169 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:13.179174 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:13.180068 (+ 894us) server_negotiation.cc:407] Connection header received
0811 19:00:13.181536 (+ 1468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:13.181540 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:13.181609 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:13.181722 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:13.183088 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:13.183762 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:13.185118 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:13.185354 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:13.185475 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:13.188074 (+ 2599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:13.188077 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:13.188079 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:13.188107 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:13.189550 (+ 1443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:13.190227 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:13.190230 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:13.190232 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:13.190321 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:13.190732 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:13.190736 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:13.190738 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:13.190946 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:13.191060 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:13.191657 (+ 597us) server_negotiation.cc:299] Negotiation successful
0811 19:00:13.191795 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 19:00:13.192884 17570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:13.179392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:40817 (local address 127.13.109.3:39811)
0811 19:00:13.179896 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:13.179942 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:13.181256 (+ 1314us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:13.181970 (+ 714us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:13.181983 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:13.182387 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:13.182919 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:13.182934 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:13.183909 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:13.183913 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:13.184975 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:13.184986 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:13.185547 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:13.185552 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:13.185736 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:13.186334 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:13.186352 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:13.187868 (+ 1516us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:00:13.189727 (+ 1859us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:13.189735 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:13.189754 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:13.190097 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:13.190420 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:13.190424 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:13.190426 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:13.190637 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:13.191061 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:00:13.191072 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:00:13.191376 (+ 304us) client_negotiation.cc:769] Sending connection context
0811 19:00:13.191589 (+ 213us) client_negotiation.cc:240] Negotiation successful
0811 19:00:13.191959 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":129,"threads_started":1}
I20260811 19:00:13.194228 17568 heartbeater.cc:347] Connected to a master server at 127.13.109.62:40817
I20260811 19:00:13.194481 17568 heartbeater.cc:464] Registering TS with master...
I20260811 19:00:13.195178 17568 heartbeater.cc:511] Master 127.13.109.62:40817 requested a full tablet report, sending...
I20260811 19:00:13.196554 17081 ts_manager.cc:194] Registered new tserver with Master: 55ed9fc42c164418bfbbaacbf2c5a200 (127.13.109.3:37607)
I20260811 19:00:13.197223 17081 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.109.3@KRBTEST.COM'} at 127.13.109.3:39811
I20260811 19:00:13.209000 13748 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 19:00:13 dist-test-slave-0kls krb5kdc[17034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474813, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 11 19:00:13 dist-test-slave-0kls krb5kdc[17034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474813, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
I20260811 19:00:13.237352 17123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:13.228665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46370 (local address 127.13.109.62:40817)
0811 19:00:13.228880 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:13.228884 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:13.228985 (+ 101us) server_negotiation.cc:407] Connection header received
0811 19:00:13.229095 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:13.229099 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:13.229147 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:13.229222 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:13.230078 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:13.230626 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:13.231543 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:13.231792 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:13.231879 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:13.234263 (+ 2384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:13.234266 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:13.234268 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:13.234297 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:13.235797 (+ 1500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:13.236176 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:13.236179 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:13.236180 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:13.236257 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:13.236484 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:13.236486 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:13.236487 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:13.236723 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:13.236829 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:13.237031 (+ 202us) server_negotiation.cc:299] Negotiation successful
0811 19:00:13.237132 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
W20260811 19:00:13.237774 17081 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:46370
I20260811 19:00:13.238847 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 17149
I20260811 19:00:13.246773 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 17290
I20260811 19:00:13.254484 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 17433
I20260811 19:00:13.263227 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 17048
2026-08-11T19:00:13Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2346 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:00:13 dist-test-slave-0kls krb5kdc[17581](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:00:13 dist-test-slave-0kls krb5kdc[17581](info): set up 2 sockets
Aug 11 19:00:13 dist-test-slave-0kls krb5kdc[17581](info): commencing operation
krb5kdc: starting...
W20260811 19:00:14.368954 13748 mini_kdc.cc:121] Time spent starting KDC: real 1.080s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/krb5kdc/test-user.keytab.
Aug 11 19:00:14 dist-test-slave-0kls krb5kdc[17581](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474814, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:00:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:00:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.109.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.109.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.109.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab.
Entry for principal kudu/127.13.109.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab.
I20260811 19:00:14.501114 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.62:37043
--webserver_interface=127.13.109.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab
--principal=kudu/127.13.109.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:39889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.109.62:37043
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:00:14.619290 17595 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:00:14.619699 17595 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:00:14.619823 17595 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:00:14.624356 17595 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:00:14.624470 17595 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:00:14.624513 17595 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:00:14.624729 17595 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:00:14.624826 17595 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:00:14.630867 17595 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:39889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.109.62:37043
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab
--principal=kudu/127.13.109.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.62:37043
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.109.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17595
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:00:14.632182 17595 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:00:14.633368 17595 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:00:14.642645 17603 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:14.642645 17600 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:14.642721 17601 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:14.643124 17595 server_base.cc:1034] running on GCE node
I20260811 19:00:14.643831 17595 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:00:14.645149 17595 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:00:14.646355 17595 hybrid_clock.cc:648] HybridClock initialized: now 1786474814646339 us; error 40 us; skew 500 ppm
Aug 11 19:00:14 dist-test-slave-0kls krb5kdc[17581](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474814, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:00:14.649484 17595 init.cc:377] Logged in from keytab as kudu/127.13.109.62@KRBTEST.COM (short username kudu)
I20260811 19:00:14.650651 17595 webserver.cc:460] Webserver started at http://127.13.109.62:35025/ using document root <none> and password file <none>
I20260811 19:00:14.651682 17595 fs_manager.cc:359] Metadata directory not provided
I20260811 19:00:14.651813 17595 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:00:14.652172 17595 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:00:14.654664 17595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/minicluster-data/master-0/data/instance:
uuid: "493bfc8321a9407885c4d95524204888"
format_stamp: "Formatted at 2026-08-11 19:00:14 on dist-test-slave-0kls"
server_key: "a4e4af33459904cb03098aa4d9336ccc"
server_key_iv: "4641569819302ec246e8dd6aeafe6274"
server_key_version: "encryptionkey@0"
I20260811 19:00:14.655326 17595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/minicluster-data/master-0/wal/instance:
uuid: "493bfc8321a9407885c4d95524204888"
format_stamp: "Formatted at 2026-08-11 19:00:14 on dist-test-slave-0kls"
server_key: "a4e4af33459904cb03098aa4d9336ccc"
server_key_iv: "4641569819302ec246e8dd6aeafe6274"
server_key_version: "encryptionkey@0"
I20260811 19:00:14.659956 17595 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 19:00:14.663125 17611 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:14.664534 17595 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 19:00:14.664710 17595 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/minicluster-data/master-0/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/minicluster-data/master-0/wal
uuid: "493bfc8321a9407885c4d95524204888"
format_stamp: "Formatted at 2026-08-11 19:00:14 on dist-test-slave-0kls"
server_key: "a4e4af33459904cb03098aa4d9336ccc"
server_key_iv: "4641569819302ec246e8dd6aeafe6274"
server_key_version: "encryptionkey@0"
I20260811 19:00:14.664837 17595 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:00:14.686535 17595 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:00:14.691248 17595 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:00:14.691504 17595 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:00:14.702160 17595 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.62:37043
I20260811 19:00:14.702207 17668 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.62:37043 every 8 connection(s)
I20260811 19:00:14.703588 17595 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
I20260811 19:00:14.707685 17669 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:00:14.709506 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 17595
I20260811 19:00:14.709663 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/minicluster-data/master-0/wal/instance
I20260811 19:00:14.710115 13748 external_mini_cluster.cc:1428] Setting key 8ece85196fb32ee12923a08ef31946e6
I20260811 19:00:14.716235 17669 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 493bfc8321a9407885c4d95524204888: Bootstrap starting.
I20260811 19:00:14.719386 17669 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 493bfc8321a9407885c4d95524204888: Neither blocks nor log segments found. Creating new log.
I20260811 19:00:14.720376 17669 log.cc:824] T 00000000000000000000000000000000 P 493bfc8321a9407885c4d95524204888: Log is configured to *not* fsync() on all Append() calls
Aug 11 19:00:14 dist-test-slave-0kls krb5kdc[17581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474814, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
I20260811 19:00:14.723060 17669 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 493bfc8321a9407885c4d95524204888: No bootstrap required, opened a new log
I20260811 19:00:14.728281 17669 raft_consensus.cc:348] T 00000000000000000000000000000000 P 493bfc8321a9407885c4d95524204888 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "493bfc8321a9407885c4d95524204888" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 37043 } }
I20260811 19:00:14.728649 17669 raft_consensus.cc:374] T 00000000000000000000000000000000 P 493bfc8321a9407885c4d95524204888 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:00:14.728747 17669 raft_consensus.cc:729] T 00000000000000000000000000000000 P 493bfc8321a9407885c4d95524204888 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 493bfc8321a9407885c4d95524204888, State: Initialized, Role: FOLLOWER
I20260811 19:00:14.729307 17669 consensus_queue.cc:260] T 00000000000000000000000000000000 P 493bfc8321a9407885c4d95524204888 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "493bfc8321a9407885c4d95524204888" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 37043 } }
I20260811 19:00:14.729450 17672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:14.711667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38476 (local address 127.13.109.62:37043)
0811 19:00:14.712121 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:14.712146 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:14.712205 (+ 59us) server_negotiation.cc:407] Connection header received
0811 19:00:14.713043 (+ 838us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:14.713069 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:14.713555 (+ 486us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:14.714136 (+ 581us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:14.715214 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:14.717021 (+ 1807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:14.718628 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:14.719498 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:14.720237 (+ 739us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:14.723041 (+ 2804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:14.723050 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:14.723074 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:14.723116 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:14.725554 (+ 2438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:14.726199 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:14.726206 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:14.726212 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:14.726360 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:14.726694 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:14.726697 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:14.726699 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:14.727285 (+ 586us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:14.727535 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:14.727916 (+ 381us) server_negotiation.cc:299] Negotiation successful
0811 19:00:14.728251 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":286,"thread_start_us":132,"threads_started":1}
I20260811 19:00:14.729482 17669 raft_consensus.cc:388] T 00000000000000000000000000000000 P 493bfc8321a9407885c4d95524204888 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:00:14.729543 17669 raft_consensus.cc:482] T 00000000000000000000000000000000 P 493bfc8321a9407885c4d95524204888 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:00:14.729635 17669 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 493bfc8321a9407885c4d95524204888 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:00:14.730969 17669 raft_consensus.cc:504] T 00000000000000000000000000000000 P 493bfc8321a9407885c4d95524204888 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "493bfc8321a9407885c4d95524204888" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 37043 } }
I20260811 19:00:14.731359 17669 leader_election.cc:302] T 00000000000000000000000000000000 P 493bfc8321a9407885c4d95524204888 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 493bfc8321a9407885c4d95524204888; no voters:
I20260811 19:00:14.731729 17669 leader_election.cc:288] T 00000000000000000000000000000000 P 493bfc8321a9407885c4d95524204888 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:00:14.731853 17675 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 493bfc8321a9407885c4d95524204888 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:00:14.732122 17675 raft_consensus.cc:686] T 00000000000000000000000000000000 P 493bfc8321a9407885c4d95524204888 [term 1 LEADER]: Becoming Leader. State: Replica: 493bfc8321a9407885c4d95524204888, State: Running, Role: LEADER
I20260811 19:00:14.732455 17675 consensus_queue.cc:237] T 00000000000000000000000000000000 P 493bfc8321a9407885c4d95524204888 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "493bfc8321a9407885c4d95524204888" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 37043 } }
I20260811 19:00:14.733111 17669 sys_catalog.cc:564] T 00000000000000000000000000000000 P 493bfc8321a9407885c4d95524204888 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:00:14.733696 17675 sys_catalog.cc:455] T 00000000000000000000000000000000 P 493bfc8321a9407885c4d95524204888 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 493bfc8321a9407885c4d95524204888. Latest consensus state: current_term: 1 leader_uuid: "493bfc8321a9407885c4d95524204888" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "493bfc8321a9407885c4d95524204888" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 37043 } } }
I20260811 19:00:14.733939 17675 sys_catalog.cc:458] T 00000000000000000000000000000000 P 493bfc8321a9407885c4d95524204888 [sys.catalog]: This master's current role is: LEADER
I20260811 19:00:14.734295 17676 sys_catalog.cc:455] T 00000000000000000000000000000000 P 493bfc8321a9407885c4d95524204888 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "493bfc8321a9407885c4d95524204888" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "493bfc8321a9407885c4d95524204888" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 37043 } } }
I20260811 19:00:14.734409 17676 sys_catalog.cc:458] T 00000000000000000000000000000000 P 493bfc8321a9407885c4d95524204888 [sys.catalog]: This master's current role is: LEADER
I20260811 19:00:14.734885 17681 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:00:14.738287 17681 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:00:14.744561 17681 catalog_manager.cc:1347] Generated new cluster ID: 7b64e95ea6ab474ea699fb740816cc35
I20260811 19:00:14.744655 17681 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:00:14.757207 17681 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:00:14.758237 17681 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:00:14.764659 17681 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 493bfc8321a9407885c4d95524204888: Generated new TSK 0
I20260811 19:00:14.765376 17681 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.109.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.109.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.109.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.1.keytab.
Entry for principal kudu/127.13.109.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.1.keytab.
I20260811 19:00:14.806478 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.1:0
--local_ip_for_outbound_sockets=127.13.109.1
--webserver_interface=127.13.109.1
--webserver_port=0
--tserver_master_addrs=127.13.109.62:37043
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.1.keytab
--principal=kudu/127.13.109.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:39889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:00:14.953604 17696 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:00:14.954023 17696 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:00:14.954085 17696 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:00:14.958250 17696 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:00:14.958324 17696 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:00:14.958388 17696 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.1
I20260811 19:00:14.964062 17696 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:39889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.1.keytab
--principal=kudu/127.13.109.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.109.1
--webserver_port=0
--tserver_master_addrs=127.13.109.62:37043
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17696
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.1
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:00:14.965245 17696 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:00:14.966338 17696 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:00:14.977159 17703 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:14.977344 17696 server_base.cc:1034] running on GCE node
W20260811 19:00:14.977149 17705 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:14.977145 17702 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:14.977977 17696 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:00:14.978648 17696 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:00:14.979816 17696 hybrid_clock.cc:648] HybridClock initialized: now 1786474814979776 us; error 53 us; skew 500 ppm
Aug 11 19:00:14 dist-test-slave-0kls krb5kdc[17581](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474814, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:00:14.982793 17696 init.cc:377] Logged in from keytab as kudu/127.13.109.1@KRBTEST.COM (short username kudu)
I20260811 19:00:14.983795 17696 webserver.cc:460] Webserver started at http://127.13.109.1:45179/ using document root <none> and password file <none>
I20260811 19:00:14.984431 17696 fs_manager.cc:359] Metadata directory not provided
I20260811 19:00:14.984498 17696 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:00:14.984742 17696 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:00:14.987005 17696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/minicluster-data/ts-0/data/instance:
uuid: "a3b7af59e1024a948ade55ae7f305e31"
format_stamp: "Formatted at 2026-08-11 19:00:14 on dist-test-slave-0kls"
server_key: "35a9f23a2077b164d7f4d1dda0e997a9"
server_key_iv: "86cdd1382006ad1e884f68157e456c6a"
server_key_version: "encryptionkey@0"
I20260811 19:00:14.987607 17696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/minicluster-data/ts-0/wal/instance:
uuid: "a3b7af59e1024a948ade55ae7f305e31"
format_stamp: "Formatted at 2026-08-11 19:00:14 on dist-test-slave-0kls"
server_key: "35a9f23a2077b164d7f4d1dda0e997a9"
server_key_iv: "86cdd1382006ad1e884f68157e456c6a"
server_key_version: "encryptionkey@0"
I20260811 19:00:14.991598 17696 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 19:00:14.994154 17712 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:14.995465 17696 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 19:00:14.995622 17696 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/minicluster-data/ts-0/wal
uuid: "a3b7af59e1024a948ade55ae7f305e31"
format_stamp: "Formatted at 2026-08-11 19:00:14 on dist-test-slave-0kls"
server_key: "35a9f23a2077b164d7f4d1dda0e997a9"
server_key_iv: "86cdd1382006ad1e884f68157e456c6a"
server_key_version: "encryptionkey@0"
I20260811 19:00:14.995749 17696 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:00:15.037345 17696 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:00:15.041260 17696 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:00:15.041509 17696 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:00:15.042272 17696 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:00:15.043256 17696 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:00:15.043340 17696 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:15.043418 17696 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:00:15.043470 17696 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:15.053685 17696 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.1:33003
I20260811 19:00:15.053714 17835 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.1:33003 every 8 connection(s)
I20260811 19:00:15.054842 17696 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
I20260811 19:00:15.056701 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 17696
I20260811 19:00:15.056852 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/minicluster-data/ts-0/wal/instance
I20260811 19:00:15.057266 13748 external_mini_cluster.cc:1428] Setting key 1f83d8100a5d9b4efddefbf78ac3bd83
Aug 11 19:00:15 dist-test-slave-0kls krb5kdc[17581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474814, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.1@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.109.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.109.2@KRBTEST.COM" created.
I20260811 19:00:15.071516 17672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:15.056915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.1:35201 (local address 127.13.109.62:37043)
0811 19:00:15.057091 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:15.057097 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:15.058217 (+ 1120us) server_negotiation.cc:407] Connection header received
0811 19:00:15.059765 (+ 1548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:15.059772 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:15.059842 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:15.059940 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:15.061310 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:15.061946 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:15.063233 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:15.063495 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:15.063623 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:15.066823 (+ 3200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:15.066829 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:15.066835 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:15.066877 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:15.068764 (+ 1887us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:15.069526 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:15.069529 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:15.069530 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:15.069629 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:15.070060 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:15.070063 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:15.070064 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:15.070401 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:15.070493 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:15.071160 (+ 667us) server_negotiation.cc:299] Negotiation successful
0811 19:00:15.071325 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 19:00:15.072474 17838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:15.057440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:37043 (local address 127.13.109.1:35201)
0811 19:00:15.058042 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:15.058106 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:15.059514 (+ 1408us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:15.060120 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:15.060128 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:15.060542 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:15.061098 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:15.061114 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:15.062104 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:15.062107 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:15.063054 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:15.063066 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:15.063687 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:15.063692 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:15.063907 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:15.064617 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:15.064636 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:15.066561 (+ 1925us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:00:15.068950 (+ 2389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:15.068959 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:15.068983 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:15.069408 (+ 425us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:15.069734 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:15.069736 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:15.069738 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:15.069969 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:15.070515 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:00:15.070521 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:00:15.070874 (+ 353us) client_negotiation.cc:769] Sending connection context
0811 19:00:15.071094 (+ 220us) client_negotiation.cc:240] Negotiation successful
0811 19:00:15.071498 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":166,"threads_started":1}
I20260811 19:00:15.074131 17836 heartbeater.cc:347] Connected to a master server at 127.13.109.62:37043
I20260811 19:00:15.074465 17836 heartbeater.cc:464] Registering TS with master...
I20260811 19:00:15.075184 17836 heartbeater.cc:511] Master 127.13.109.62:37043 requested a full tablet report, sending...
I20260811 19:00:15.077195 17630 ts_manager.cc:194] Registered new tserver with Master: a3b7af59e1024a948ade55ae7f305e31 (127.13.109.1:33003)
I20260811 19:00:15.079078 17630 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.109.1@KRBTEST.COM'} at 127.13.109.1:35201
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.109.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.2.keytab.
Entry for principal kudu/127.13.109.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.2.keytab.
I20260811 19:00:15.089287 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.2:0
--local_ip_for_outbound_sockets=127.13.109.2
--webserver_interface=127.13.109.2
--webserver_port=0
--tserver_master_addrs=127.13.109.62:37043
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.2.keytab
--principal=kudu/127.13.109.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:39889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:00:15.212774 17841 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:00:15.213101 17841 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:00:15.213150 17841 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:00:15.217769 17841 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:00:15.217890 17841 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:00:15.217995 17841 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.2
I20260811 19:00:15.224164 17841 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:39889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.2.keytab
--principal=kudu/127.13.109.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.109.2
--webserver_port=0
--tserver_master_addrs=127.13.109.62:37043
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17841
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.2
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:00:15.225420 17841 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:00:15.226691 17841 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:00:15.237701 17849 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:15.237697 17847 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:15.237757 17841 server_base.cc:1034] running on GCE node
W20260811 19:00:15.237663 17846 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:15.238528 17841 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:00:15.239327 17841 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:00:15.240542 17841 hybrid_clock.cc:648] HybridClock initialized: now 1786474815240519 us; error 48 us; skew 500 ppm
Aug 11 19:00:15 dist-test-slave-0kls krb5kdc[17581](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474815, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:00:15.243707 17841 init.cc:377] Logged in from keytab as kudu/127.13.109.2@KRBTEST.COM (short username kudu)
I20260811 19:00:15.244799 17841 webserver.cc:460] Webserver started at http://127.13.109.2:45127/ using document root <none> and password file <none>
I20260811 19:00:15.245462 17841 fs_manager.cc:359] Metadata directory not provided
I20260811 19:00:15.245584 17841 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:00:15.245945 17841 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:00:15.248342 17841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/minicluster-data/ts-1/data/instance:
uuid: "6d47392d2fc94b77a057df188cd299fe"
format_stamp: "Formatted at 2026-08-11 19:00:15 on dist-test-slave-0kls"
server_key: "71ba74818beaa3253281a8a9d2dc3b2e"
server_key_iv: "4128543456d879597bb26053f01599c6"
server_key_version: "encryptionkey@0"
I20260811 19:00:15.249037 17841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/minicluster-data/ts-1/wal/instance:
uuid: "6d47392d2fc94b77a057df188cd299fe"
format_stamp: "Formatted at 2026-08-11 19:00:15 on dist-test-slave-0kls"
server_key: "71ba74818beaa3253281a8a9d2dc3b2e"
server_key_iv: "4128543456d879597bb26053f01599c6"
server_key_version: "encryptionkey@0"
I20260811 19:00:15.253582 17841 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260811 19:00:15.256410 17856 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:15.257719 17841 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 19:00:15.257916 17841 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/minicluster-data/ts-1/wal
uuid: "6d47392d2fc94b77a057df188cd299fe"
format_stamp: "Formatted at 2026-08-11 19:00:15 on dist-test-slave-0kls"
server_key: "71ba74818beaa3253281a8a9d2dc3b2e"
server_key_iv: "4128543456d879597bb26053f01599c6"
server_key_version: "encryptionkey@0"
I20260811 19:00:15.258061 17841 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:00:15.280097 17841 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:00:15.283725 17841 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:00:15.283954 17841 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:00:15.284622 17841 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:00:15.285542 17841 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:00:15.285617 17841 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:15.285696 17841 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:00:15.285748 17841 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:15.294876 17841 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.2:43243
I20260811 19:00:15.294898 17974 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.2:43243 every 8 connection(s)
I20260811 19:00:15.295840 17841 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
I20260811 19:00:15.298656 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 17841
I20260811 19:00:15.298794 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/minicluster-data/ts-1/wal/instance
I20260811 19:00:15.299127 13748 external_mini_cluster.cc:1428] Setting key 5b905eaba1c0890f18ab8283f8f61104
Aug 11 19:00:15 dist-test-slave-0kls krb5kdc[17581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474815, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.2@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.109.3@KRBTEST.COM; defaulting to no policy
I20260811 19:00:15.312388 17672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:15.297715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.2:39449 (local address 127.13.109.62:37043)
0811 19:00:15.297883 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:15.297887 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:15.298841 (+ 954us) server_negotiation.cc:407] Connection header received
0811 19:00:15.300767 (+ 1926us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:15.300771 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:15.300826 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:15.300890 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:15.302640 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:15.303526 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:15.304771 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:15.305182 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:15.305321 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:15.308306 (+ 2985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:15.308311 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:15.308314 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:15.308350 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:15.310143 (+ 1793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:15.310720 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:15.310725 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:15.310729 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:15.310839 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:15.311217 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:15.311220 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:15.311221 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:15.311424 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:15.311507 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:15.312132 (+ 625us) server_negotiation.cc:299] Negotiation successful
0811 19:00:15.312255 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 19:00:15.313333 17977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:15.298117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:37043 (local address 127.13.109.2:39449)
0811 19:00:15.298655 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:15.298718 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:15.300553 (+ 1835us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:15.301077 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:15.301086 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:15.301648 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:15.302434 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:15.302459 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:15.303706 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:15.303709 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:15.304629 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:15.304640 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:15.305358 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:15.305362 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:15.305521 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:15.306280 (+ 759us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:15.306311 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:15.308092 (+ 1781us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:00:15.310273 (+ 2181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:15.310279 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:15.310295 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:15.310583 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:15.310944 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:15.310947 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:15.310948 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:15.311100 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:15.311519 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:00:15.311524 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:00:15.311811 (+ 287us) client_negotiation.cc:769] Sending connection context
0811 19:00:15.312007 (+ 196us) client_negotiation.cc:240] Negotiation successful
0811 19:00:15.312300 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":126,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.109.3@KRBTEST.COM" created.
I20260811 19:00:15.314735 17975 heartbeater.cc:347] Connected to a master server at 127.13.109.62:37043
I20260811 19:00:15.314989 17975 heartbeater.cc:464] Registering TS with master...
I20260811 19:00:15.315537 17975 heartbeater.cc:511] Master 127.13.109.62:37043 requested a full tablet report, sending...
I20260811 19:00:15.316669 17630 ts_manager.cc:194] Registered new tserver with Master: 6d47392d2fc94b77a057df188cd299fe (127.13.109.2:43243)
I20260811 19:00:15.317512 17630 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.109.2@KRBTEST.COM'} at 127.13.109.2:39449
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.109.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.3.keytab.
Entry for principal kudu/127.13.109.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.3.keytab.
I20260811 19:00:15.333477 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.3:0
--local_ip_for_outbound_sockets=127.13.109.3
--webserver_interface=127.13.109.3
--webserver_port=0
--tserver_master_addrs=127.13.109.62:37043
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.3.keytab
--principal=kudu/127.13.109.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:39889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:00:15.457998 17980 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:00:15.458276 17980 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:00:15.458320 17980 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:00:15.462572 17980 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:00:15.462671 17980 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:00:15.462734 17980 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.3
I20260811 19:00:15.468111 17980 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:39889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.3.keytab
--principal=kudu/127.13.109.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.109.3
--webserver_port=0
--tserver_master_addrs=127.13.109.62:37043
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17980
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.3
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:00:15.469333 17980 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:00:15.470494 17980 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:00:15.481045 17985 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:15.481035 17986 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:15.481122 17980 server_base.cc:1034] running on GCE node
W20260811 19:00:15.481060 17988 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:15.481734 17980 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:00:15.482455 17980 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:00:15.483646 17980 hybrid_clock.cc:648] HybridClock initialized: now 1786474815483633 us; error 26 us; skew 500 ppm
Aug 11 19:00:15 dist-test-slave-0kls krb5kdc[17581](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474815, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:00:15.486692 17980 init.cc:377] Logged in from keytab as kudu/127.13.109.3@KRBTEST.COM (short username kudu)
I20260811 19:00:15.487670 17980 webserver.cc:460] Webserver started at http://127.13.109.3:42987/ using document root <none> and password file <none>
I20260811 19:00:15.488251 17980 fs_manager.cc:359] Metadata directory not provided
I20260811 19:00:15.488314 17980 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:00:15.488497 17980 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:00:15.490778 17980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/minicluster-data/ts-2/data/instance:
uuid: "133fc446f72448878e4e505915dd9229"
format_stamp: "Formatted at 2026-08-11 19:00:15 on dist-test-slave-0kls"
server_key: "c90c949582a3b85d1d53a419053209b7"
server_key_iv: "97356d4c30b45a550e29f0e0a2b78c32"
server_key_version: "encryptionkey@0"
I20260811 19:00:15.491385 17980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/minicluster-data/ts-2/wal/instance:
uuid: "133fc446f72448878e4e505915dd9229"
format_stamp: "Formatted at 2026-08-11 19:00:15 on dist-test-slave-0kls"
server_key: "c90c949582a3b85d1d53a419053209b7"
server_key_iv: "97356d4c30b45a550e29f0e0a2b78c32"
server_key_version: "encryptionkey@0"
I20260811 19:00:15.495563 17980 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 19:00:15.498421 17996 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:15.499832 17980 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 19:00:15.499980 17980 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/minicluster-data/ts-2/wal
uuid: "133fc446f72448878e4e505915dd9229"
format_stamp: "Formatted at 2026-08-11 19:00:15 on dist-test-slave-0kls"
server_key: "c90c949582a3b85d1d53a419053209b7"
server_key_iv: "97356d4c30b45a550e29f0e0a2b78c32"
server_key_version: "encryptionkey@0"
I20260811 19:00:15.500119 17980 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:00:15.514384 17980 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:00:15.518049 17980 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:00:15.518275 17980 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:00:15.518939 17980 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:00:15.519876 17980 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:00:15.520012 17980 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:15.520089 17980 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:00:15.520143 17980 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:15.529659 17980 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.3:32969
I20260811 19:00:15.529680 18115 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.3:32969 every 8 connection(s)
I20260811 19:00:15.530772 17980 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
I20260811 19:00:15.532343 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 17980
I20260811 19:00:15.532487 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474792496492-13748-0/minicluster-data/ts-2/wal/instance
I20260811 19:00:15.532835 13748 external_mini_cluster.cc:1428] Setting key e326bebfa889927737798e332f18239d
Aug 11 19:00:15 dist-test-slave-0kls krb5kdc[17581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474815, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.3@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
I20260811 19:00:15.548934 17672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:15.532897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.3:44119 (local address 127.13.109.62:37043)
0811 19:00:15.533048 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:15.533071 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:15.534395 (+ 1324us) server_negotiation.cc:407] Connection header received
0811 19:00:15.536186 (+ 1791us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:15.536189 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:15.536254 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:15.536329 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:15.537638 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:15.538274 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:15.539455 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:15.539695 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:15.539826 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:15.543876 (+ 4050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:15.543881 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:15.543883 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:15.543924 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:15.546500 (+ 2576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:15.547246 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:15.547249 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:15.547251 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:15.547346 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:15.547734 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:15.547739 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:15.547741 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:15.547950 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:15.548065 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:15.548630 (+ 565us) server_negotiation.cc:299] Negotiation successful
0811 19:00:15.548765 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 19:00:15.549997 18118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:15.533525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:37043 (local address 127.13.109.3:44119)
0811 19:00:15.534226 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:15.534291 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:15.535933 (+ 1642us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:15.536529 (+ 596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:15.536537 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:15.536946 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:15.537474 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:15.537488 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:15.538412 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:15.538416 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:15.539318 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:15.539328 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:15.539871 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:15.539875 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:15.540064 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:15.540642 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:15.540659 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:15.543532 (+ 2873us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:00:15.546668 (+ 3136us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:15.546680 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:15.546708 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:15.547095 (+ 387us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:15.547450 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:15.547454 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:15.547456 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:15.547629 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:15.548066 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:00:15.548075 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:00:15.548372 (+ 297us) client_negotiation.cc:769] Sending connection context
0811 19:00:15.548583 (+ 211us) client_negotiation.cc:240] Negotiation successful
0811 19:00:15.548967 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":169,"threads_started":1}
I20260811 19:00:15.551231 18116 heartbeater.cc:347] Connected to a master server at 127.13.109.62:37043
I20260811 19:00:15.551505 18116 heartbeater.cc:464] Registering TS with master...
I20260811 19:00:15.552279 18116 heartbeater.cc:511] Master 127.13.109.62:37043 requested a full tablet report, sending...
I20260811 19:00:15.553376 17630 ts_manager.cc:194] Registered new tserver with Master: 133fc446f72448878e4e505915dd9229 (127.13.109.3:32969)
I20260811 19:00:15.554032 17630 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.109.3@KRBTEST.COM'} at 127.13.109.3:44119
I20260811 19:00:15.559660 13748 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 19:00:15 dist-test-slave-0kls krb5kdc[17581](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474815, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 19:00:15 dist-test-slave-0kls krb5kdc[17581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474815, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.109.1@KRBTEST.COM
I20260811 19:00:15.588968 18124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:15.579253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44846 (local address 127.13.109.1:33003)
0811 19:00:15.579575 (+ 322us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:15.579581 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:15.579755 (+ 174us) server_negotiation.cc:407] Connection header received
0811 19:00:15.579864 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:15.579870 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:15.580066 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:15.580188 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:15.580837 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:15.581639 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:15.582512 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:15.582898 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:15.582987 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:15.585448 (+ 2461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:15.585458 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:15.585474 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:15.585499 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:15.587129 (+ 1630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:15.587568 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:15.587573 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:15.587575 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:15.587684 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:15.587975 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:15.587978 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:15.587980 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:15.588288 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:15.588454 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:15.588643 (+ 189us) server_negotiation.cc:299] Negotiation successful
0811 19:00:15.588774 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":101,"threads_started":1}
Aug 11 19:00:15 dist-test-slave-0kls krb5kdc[17581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474815, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
I20260811 19:00:15.600524 17672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:15.591977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38478 (local address 127.13.109.62:37043)
0811 19:00:15.592121 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:15.592126 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:15.592209 (+ 83us) server_negotiation.cc:407] Connection header received
0811 19:00:15.592364 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:15.592367 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:15.592418 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:15.592503 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:15.593104 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:15.593627 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:15.594483 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:15.594720 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:15.594845 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:15.597228 (+ 2383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:15.597233 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:15.597235 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:15.597268 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:15.598769 (+ 1501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:15.599198 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:15.599201 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:15.599203 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:15.599291 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:15.599574 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:15.599577 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:15.599579 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:15.599836 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:15.599979 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:15.600225 (+ 246us) server_negotiation.cc:299] Negotiation successful
0811 19:00:15.600354 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
W20260811 19:00:15.600984 17630 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38478
I20260811 19:00:15.601665 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 17696
I20260811 19:00:15.612712 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 17841
I20260811 19:00:15.621110 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 17980
I20260811 19:00:15.631728 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 17595
2026-08-11T19:00:15Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2367 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:00:15 dist-test-slave-0kls krb5kdc[18129](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:00:15 dist-test-slave-0kls krb5kdc[18129](info): set up 2 sockets
Aug 11 19:00:15 dist-test-slave-0kls krb5kdc[18129](info): commencing operation
krb5kdc: starting...
W20260811 19:00:16.728354 13748 mini_kdc.cc:121] Time spent starting KDC: real 1.072s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/krb5kdc/test-user.keytab.
Aug 11 19:00:16 dist-test-slave-0kls krb5kdc[18129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474816, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:00:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:00:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.109.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.109.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.109.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab.
Entry for principal kudu/127.13.109.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab.
I20260811 19:00:16.861218 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.62:46061
--webserver_interface=127.13.109.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab
--principal=kudu/127.13.109.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:43747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.109.62:46061
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:00:16.979414 18142 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:00:16.979717 18142 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:00:16.979782 18142 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:00:16.983809 18142 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:00:16.983881 18142 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:00:16.983901 18142 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:00:16.983916 18142 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:00:16.983932 18142 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:00:16.989342 18142 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:43747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.109.62:46061
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab
--principal=kudu/127.13.109.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.62:46061
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.109.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18142
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:00:16.990514 18142 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:00:16.991631 18142 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:00:17.000722 18147 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:17.000712 18150 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:17.000806 18142 server_base.cc:1034] running on GCE node
W20260811 19:00:17.000715 18148 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:17.001847 18142 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:00:17.003151 18142 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:00:17.004357 18142 hybrid_clock.cc:648] HybridClock initialized: now 1786474817004340 us; error 34 us; skew 500 ppm
Aug 11 19:00:17 dist-test-slave-0kls krb5kdc[18129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474817, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:00:17.012240 18142 init.cc:377] Logged in from keytab as kudu/127.13.109.62@KRBTEST.COM (short username kudu)
I20260811 19:00:17.013159 18142 fs_manager.cc:359] Metadata directory not provided
I20260811 19:00:17.013278 18142 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:00:17.013556 18142 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:00:17.015952 18142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/minicluster-data/master-0/data/instance:
uuid: "e1102f5c3f804fc7947b5b44dd58c9b4"
format_stamp: "Formatted at 2026-08-11 19:00:17 on dist-test-slave-0kls"
server_key: "f3d70a9696447be54abd8ed02bf81e00"
server_key_iv: "b1c97ff1fcd856880f7456ee4410d3af"
server_key_version: "encryptionkey@0"
I20260811 19:00:17.016588 18142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/minicluster-data/master-0/wal/instance:
uuid: "e1102f5c3f804fc7947b5b44dd58c9b4"
format_stamp: "Formatted at 2026-08-11 19:00:17 on dist-test-slave-0kls"
server_key: "f3d70a9696447be54abd8ed02bf81e00"
server_key_iv: "b1c97ff1fcd856880f7456ee4410d3af"
server_key_version: "encryptionkey@0"
I20260811 19:00:17.020727 18142 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260811 19:00:17.023839 18156 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:17.025242 18142 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 19:00:17.025416 18142 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/minicluster-data/master-0/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/minicluster-data/master-0/wal
uuid: "e1102f5c3f804fc7947b5b44dd58c9b4"
format_stamp: "Formatted at 2026-08-11 19:00:17 on dist-test-slave-0kls"
server_key: "f3d70a9696447be54abd8ed02bf81e00"
server_key_iv: "b1c97ff1fcd856880f7456ee4410d3af"
server_key_version: "encryptionkey@0"
I20260811 19:00:17.025559 18142 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:00:17.044081 18142 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:00:17.047641 18142 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:00:17.047888 18142 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:00:17.055413 18142 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.62:46061
I20260811 19:00:17.055759 18211 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.62:46061 every 8 connection(s)
I20260811 19:00:17.056255 18142 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
I20260811 19:00:17.059351 18212 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:00:17.059617 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 18142
I20260811 19:00:17.059793 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/minicluster-data/master-0/wal/instance
I20260811 19:00:17.060081 13748 external_mini_cluster.cc:1428] Setting key d9fd20bcbc6e51cf6097a4fa01d2342a
I20260811 19:00:17.067643 18212 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e1102f5c3f804fc7947b5b44dd58c9b4: Bootstrap starting.
Aug 11 19:00:17 dist-test-slave-0kls krb5kdc[18129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474816, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
I20260811 19:00:17.070564 18212 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e1102f5c3f804fc7947b5b44dd58c9b4: Neither blocks nor log segments found. Creating new log.
I20260811 19:00:17.071669 18212 log.cc:824] T 00000000000000000000000000000000 P e1102f5c3f804fc7947b5b44dd58c9b4: Log is configured to *not* fsync() on all Append() calls
I20260811 19:00:17.074522 18212 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e1102f5c3f804fc7947b5b44dd58c9b4: No bootstrap required, opened a new log
I20260811 19:00:17.076886 18215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:17.061600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55796 (local address 127.13.109.62:46061)
0811 19:00:17.062120 (+ 520us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:17.062135 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:17.062175 (+ 40us) server_negotiation.cc:407] Connection header received
0811 19:00:17.062806 (+ 631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:17.062822 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:17.063195 (+ 373us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:17.063558 (+ 363us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:17.064568 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:17.066175 (+ 1607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:17.067102 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:17.067829 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:17.068001 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:17.070893 (+ 2892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:17.070903 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:17.070928 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:17.070973 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:17.072945 (+ 1972us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:17.073440 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:17.073449 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:17.073457 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:17.073620 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:17.073961 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:17.073964 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:17.073965 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:17.074507 (+ 542us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:17.074740 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:17.075193 (+ 453us) server_negotiation.cc:299] Negotiation successful
0811 19:00:17.075532 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":194,"threads_started":1}
I20260811 19:00:17.078823 18212 raft_consensus.cc:348] T 00000000000000000000000000000000 P e1102f5c3f804fc7947b5b44dd58c9b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1102f5c3f804fc7947b5b44dd58c9b4" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 46061 } }
I20260811 19:00:17.079118 18212 raft_consensus.cc:374] T 00000000000000000000000000000000 P e1102f5c3f804fc7947b5b44dd58c9b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:00:17.079212 18212 raft_consensus.cc:729] T 00000000000000000000000000000000 P e1102f5c3f804fc7947b5b44dd58c9b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1102f5c3f804fc7947b5b44dd58c9b4, State: Initialized, Role: FOLLOWER
I20260811 19:00:17.079717 18212 consensus_queue.cc:260] T 00000000000000000000000000000000 P e1102f5c3f804fc7947b5b44dd58c9b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1102f5c3f804fc7947b5b44dd58c9b4" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 46061 } }
I20260811 19:00:17.079874 18212 raft_consensus.cc:388] T 00000000000000000000000000000000 P e1102f5c3f804fc7947b5b44dd58c9b4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:00:17.079967 18212 raft_consensus.cc:482] T 00000000000000000000000000000000 P e1102f5c3f804fc7947b5b44dd58c9b4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:00:17.080086 18212 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e1102f5c3f804fc7947b5b44dd58c9b4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:00:17.081151 18212 raft_consensus.cc:504] T 00000000000000000000000000000000 P e1102f5c3f804fc7947b5b44dd58c9b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1102f5c3f804fc7947b5b44dd58c9b4" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 46061 } }
I20260811 19:00:17.081534 18212 leader_election.cc:302] T 00000000000000000000000000000000 P e1102f5c3f804fc7947b5b44dd58c9b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e1102f5c3f804fc7947b5b44dd58c9b4; no voters:
I20260811 19:00:17.081969 18212 leader_election.cc:288] T 00000000000000000000000000000000 P e1102f5c3f804fc7947b5b44dd58c9b4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:00:17.082088 18217 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e1102f5c3f804fc7947b5b44dd58c9b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:00:17.082356 18217 raft_consensus.cc:686] T 00000000000000000000000000000000 P e1102f5c3f804fc7947b5b44dd58c9b4 [term 1 LEADER]: Becoming Leader. State: Replica: e1102f5c3f804fc7947b5b44dd58c9b4, State: Running, Role: LEADER
I20260811 19:00:17.082787 18217 consensus_queue.cc:237] T 00000000000000000000000000000000 P e1102f5c3f804fc7947b5b44dd58c9b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1102f5c3f804fc7947b5b44dd58c9b4" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 46061 } }
I20260811 19:00:17.083472 18212 sys_catalog.cc:564] T 00000000000000000000000000000000 P e1102f5c3f804fc7947b5b44dd58c9b4 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:00:17.084491 18219 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1102f5c3f804fc7947b5b44dd58c9b4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e1102f5c3f804fc7947b5b44dd58c9b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1102f5c3f804fc7947b5b44dd58c9b4" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 46061 } } }
I20260811 19:00:17.084637 18219 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1102f5c3f804fc7947b5b44dd58c9b4 [sys.catalog]: This master's current role is: LEADER
I20260811 19:00:17.085053 18220 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1102f5c3f804fc7947b5b44dd58c9b4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e1102f5c3f804fc7947b5b44dd58c9b4. Latest consensus state: current_term: 1 leader_uuid: "e1102f5c3f804fc7947b5b44dd58c9b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1102f5c3f804fc7947b5b44dd58c9b4" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 46061 } } }
I20260811 19:00:17.085251 18220 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1102f5c3f804fc7947b5b44dd58c9b4 [sys.catalog]: This master's current role is: LEADER
I20260811 19:00:17.085151 18228 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:00:17.089004 18228 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:00:17.095191 18228 catalog_manager.cc:1347] Generated new cluster ID: 82385723ffa44cd8ae3a572d8e3c91c0
I20260811 19:00:17.095284 18228 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:00:17.104002 18228 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:00:17.105136 18228 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:00:17.111780 18228 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e1102f5c3f804fc7947b5b44dd58c9b4: Generated new TSK 0
I20260811 19:00:17.112596 18228 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.109.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.109.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.109.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.1.keytab.
Entry for principal kudu/127.13.109.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.1.keytab.
I20260811 19:00:17.149359 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.1:0
--local_ip_for_outbound_sockets=127.13.109.1
--webserver_interface=127.13.109.1
--webserver_port=0
--tserver_master_addrs=127.13.109.62:46061
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.1.keytab
--principal=kudu/127.13.109.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:43747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:00:17.271028 18240 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:00:17.271385 18240 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:00:17.271474 18240 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:00:17.275870 18240 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:00:17.275998 18240 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:00:17.276105 18240 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.1
I20260811 19:00:17.282066 18240 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:43747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.1.keytab
--principal=kudu/127.13.109.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.109.1
--webserver_port=0
--tserver_master_addrs=127.13.109.62:46061
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18240
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.1
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:00:17.283370 18240 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:00:17.284597 18240 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:00:17.295032 18249 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:17.295194 18240 server_base.cc:1034] running on GCE node
W20260811 19:00:17.295032 18246 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:17.295032 18247 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:17.295785 18240 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:00:17.296543 18240 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:00:17.297741 18240 hybrid_clock.cc:648] HybridClock initialized: now 1786474817297715 us; error 37 us; skew 500 ppm
Aug 11 19:00:17 dist-test-slave-0kls krb5kdc[18129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474817, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:00:17.300647 18240 init.cc:377] Logged in from keytab as kudu/127.13.109.1@KRBTEST.COM (short username kudu)
I20260811 19:00:17.301391 18240 fs_manager.cc:359] Metadata directory not provided
I20260811 19:00:17.301456 18240 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:00:17.301635 18240 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:00:17.303861 18240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/minicluster-data/ts-0/data/instance:
uuid: "d7924d051943430495203c2582978852"
format_stamp: "Formatted at 2026-08-11 19:00:17 on dist-test-slave-0kls"
server_key: "69c7c551abc643e853aab3ab2fbea241"
server_key_iv: "2e2040e17dcf8c10dade73a193689bbc"
server_key_version: "encryptionkey@0"
I20260811 19:00:17.304435 18240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/minicluster-data/ts-0/wal/instance:
uuid: "d7924d051943430495203c2582978852"
format_stamp: "Formatted at 2026-08-11 19:00:17 on dist-test-slave-0kls"
server_key: "69c7c551abc643e853aab3ab2fbea241"
server_key_iv: "2e2040e17dcf8c10dade73a193689bbc"
server_key_version: "encryptionkey@0"
I20260811 19:00:17.308643 18240 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260811 19:00:17.311456 18256 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:17.312908 18240 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 19:00:17.313058 18240 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/minicluster-data/ts-0/wal
uuid: "d7924d051943430495203c2582978852"
format_stamp: "Formatted at 2026-08-11 19:00:17 on dist-test-slave-0kls"
server_key: "69c7c551abc643e853aab3ab2fbea241"
server_key_iv: "2e2040e17dcf8c10dade73a193689bbc"
server_key_version: "encryptionkey@0"
I20260811 19:00:17.313202 18240 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:00:17.337524 18240 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:00:17.341220 18240 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:00:17.341467 18240 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:00:17.342252 18240 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:00:17.343222 18240 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:00:17.343272 18240 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:17.343348 18240 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:00:17.343380 18240 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:17.353245 18240 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.1:43603
I20260811 19:00:17.353263 18375 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.1:43603 every 8 connection(s)
I20260811 19:00:17.354040 18240 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
I20260811 19:00:17.358258 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 18240
I20260811 19:00:17.358376 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/minicluster-data/ts-0/wal/instance
I20260811 19:00:17.358702 13748 external_mini_cluster.cc:1428] Setting key 43edef7b81ec69c2798099810594886b
Aug 11 19:00:17 dist-test-slave-0kls krb5kdc[18129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474817, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.1@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.109.2@KRBTEST.COM; defaulting to no policy
I20260811 19:00:17.369699 18215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:17.356099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.1:42967 (local address 127.13.109.62:46061)
0811 19:00:17.356259 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:17.356263 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:17.357229 (+ 966us) server_negotiation.cc:407] Connection header received
0811 19:00:17.358761 (+ 1532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:17.358765 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:17.358833 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:17.358950 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:17.360678 (+ 1728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:17.361218 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:17.362511 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:17.362758 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:17.362879 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:17.365661 (+ 2782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:17.365664 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:17.365667 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:17.365695 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:17.367298 (+ 1603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:17.367904 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:17.367908 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:17.367910 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:17.368004 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:17.368384 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:17.368389 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:17.368391 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:17.368612 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:17.368727 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:17.369341 (+ 614us) server_negotiation.cc:299] Negotiation successful
0811 19:00:17.369496 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 19:00:17.370798 18378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:17.356532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:46061 (local address 127.13.109.1:42967)
0811 19:00:17.357049 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:17.357103 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:17.358520 (+ 1417us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:17.359191 (+ 671us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:17.359202 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:17.359756 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:17.360515 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:17.360534 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:17.361357 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:17.361361 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:17.362370 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:17.362382 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:17.362932 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:17.362937 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:17.363132 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:17.363793 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:17.363812 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:17.365448 (+ 1636us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:00:17.367438 (+ 1990us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:17.367446 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:17.367468 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:17.367775 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:17.368108 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:17.368111 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:17.368114 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:17.368282 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:17.368727 (+ 445us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:00:17.368736 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:00:17.369039 (+ 303us) client_negotiation.cc:769] Sending connection context
0811 19:00:17.369266 (+ 227us) client_negotiation.cc:240] Negotiation successful
0811 19:00:17.369661 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":120,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.109.2@KRBTEST.COM" created.
I20260811 19:00:17.372301 18376 heartbeater.cc:347] Connected to a master server at 127.13.109.62:46061
I20260811 19:00:17.372610 18376 heartbeater.cc:464] Registering TS with master...
I20260811 19:00:17.373267 18376 heartbeater.cc:511] Master 127.13.109.62:46061 requested a full tablet report, sending...
I20260811 19:00:17.375144 18174 ts_manager.cc:194] Registered new tserver with Master: d7924d051943430495203c2582978852 (127.13.109.1:43603)
I20260811 19:00:17.376374 18174 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.109.1@KRBTEST.COM'} at 127.13.109.1:42967
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.109.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.2.keytab.
Entry for principal kudu/127.13.109.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.2.keytab.
I20260811 19:00:17.391067 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.2:0
--local_ip_for_outbound_sockets=127.13.109.2
--webserver_interface=127.13.109.2
--webserver_port=0
--tserver_master_addrs=127.13.109.62:46061
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.2.keytab
--principal=kudu/127.13.109.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:43747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:00:17.510931 18381 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:00:17.511215 18381 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:00:17.511261 18381 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:00:17.515246 18381 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:00:17.515313 18381 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:00:17.515375 18381 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.2
I20260811 19:00:17.521039 18381 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:43747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.2.keytab
--principal=kudu/127.13.109.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.109.2
--webserver_port=0
--tserver_master_addrs=127.13.109.62:46061
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18381
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.2
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:00:17.522253 18381 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:00:17.523455 18381 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:00:17.533950 18386 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:17.533959 18381 server_base.cc:1034] running on GCE node
W20260811 19:00:17.534127 18390 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:17.533907 18387 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:17.534724 18381 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:00:17.535471 18381 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:00:17.536634 18381 hybrid_clock.cc:648] HybridClock initialized: now 1786474817536594 us; error 48 us; skew 500 ppm
Aug 11 19:00:17 dist-test-slave-0kls krb5kdc[18129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474817, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:00:17.539482 18381 init.cc:377] Logged in from keytab as kudu/127.13.109.2@KRBTEST.COM (short username kudu)
I20260811 19:00:17.540258 18381 fs_manager.cc:359] Metadata directory not provided
I20260811 19:00:17.540355 18381 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:00:17.540596 18381 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:00:17.542796 18381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/minicluster-data/ts-1/data/instance:
uuid: "97d793a87a784870b326af48e8e6fe9b"
format_stamp: "Formatted at 2026-08-11 19:00:17 on dist-test-slave-0kls"
server_key: "88f60499d7bf9f2e7fc3605dbdf62ffb"
server_key_iv: "087fd9538220ac0c16336e143b5d4f5f"
server_key_version: "encryptionkey@0"
I20260811 19:00:17.543394 18381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/minicluster-data/ts-1/wal/instance:
uuid: "97d793a87a784870b326af48e8e6fe9b"
format_stamp: "Formatted at 2026-08-11 19:00:17 on dist-test-slave-0kls"
server_key: "88f60499d7bf9f2e7fc3605dbdf62ffb"
server_key_iv: "087fd9538220ac0c16336e143b5d4f5f"
server_key_version: "encryptionkey@0"
I20260811 19:00:17.547277 18381 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 19:00:17.549746 18397 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:17.551247 18381 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 19:00:17.551358 18381 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/minicluster-data/ts-1/wal
uuid: "97d793a87a784870b326af48e8e6fe9b"
format_stamp: "Formatted at 2026-08-11 19:00:17 on dist-test-slave-0kls"
server_key: "88f60499d7bf9f2e7fc3605dbdf62ffb"
server_key_iv: "087fd9538220ac0c16336e143b5d4f5f"
server_key_version: "encryptionkey@0"
I20260811 19:00:17.551460 18381 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:00:17.588444 18381 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:00:17.591677 18381 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:00:17.591836 18381 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:00:17.592383 18381 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:00:17.593215 18381 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:00:17.593302 18381 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:17.593384 18381 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:00:17.593425 18381 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:17.602938 18381 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.2:45071
I20260811 19:00:17.602957 18513 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.2:45071 every 8 connection(s)
I20260811 19:00:17.603549 18381 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
I20260811 19:00:17.609978 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 18381
I20260811 19:00:17.610123 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/minicluster-data/ts-1/wal/instance
I20260811 19:00:17.610464 13748 external_mini_cluster.cc:1428] Setting key a2dc2eb3fd95b50455e94a7797dc05d1
Aug 11 19:00:17 dist-test-slave-0kls krb5kdc[18129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474817, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.2@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
I20260811 19:00:17.619333 18215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:17.605534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.2:56083 (local address 127.13.109.62:46061)
0811 19:00:17.605718 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:17.605723 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:17.606682 (+ 959us) server_negotiation.cc:407] Connection header received
0811 19:00:17.608000 (+ 1318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:17.608003 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:17.608055 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:17.608140 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:17.609396 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:17.609991 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:17.611628 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:17.611926 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:17.612047 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:17.615082 (+ 3035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:17.615087 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:17.615090 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:17.615126 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:17.616642 (+ 1516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:17.617409 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:17.617412 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:17.617413 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:17.617493 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:17.617934 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:17.617937 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:17.617938 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:17.618237 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:17.618320 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:17.619085 (+ 765us) server_negotiation.cc:299] Negotiation successful
0811 19:00:17.619199 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 19:00:17.620733 18517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:17.605999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:46061 (local address 127.13.109.2:56083)
0811 19:00:17.606506 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:17.606558 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:17.607792 (+ 1234us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:17.608322 (+ 530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:17.608329 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:17.608709 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:17.609252 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:17.609266 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:17.610147 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:17.610153 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:17.611489 (+ 1336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:17.611500 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:17.612141 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:17.612147 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:17.612353 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:17.613017 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:17.613036 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:17.614881 (+ 1845us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:00:17.616777 (+ 1896us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:17.616787 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:17.616810 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:17.617291 (+ 481us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:17.617600 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:17.617606 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:17.617608 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:17.617837 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:17.618356 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:00:17.618366 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:00:17.618783 (+ 417us) client_negotiation.cc:769] Sending connection context
0811 19:00:17.619040 (+ 257us) client_negotiation.cc:240] Negotiation successful
0811 19:00:17.619477 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":124,"threads_started":1}
WARNING: no policy specified for kudu/127.13.109.3@KRBTEST.COM; defaulting to no policy
I20260811 19:00:17.622117 18514 heartbeater.cc:347] Connected to a master server at 127.13.109.62:46061
I20260811 19:00:17.622406 18514 heartbeater.cc:464] Registering TS with master...
I20260811 19:00:17.622958 18514 heartbeater.cc:511] Master 127.13.109.62:46061 requested a full tablet report, sending...
I20260811 19:00:17.624200 18174 ts_manager.cc:194] Registered new tserver with Master: 97d793a87a784870b326af48e8e6fe9b (127.13.109.2:45071)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.109.3@KRBTEST.COM" created.
I20260811 19:00:17.625123 18174 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.109.2@KRBTEST.COM'} at 127.13.109.2:56083
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.109.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.3.keytab.
Entry for principal kudu/127.13.109.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.3.keytab.
I20260811 19:00:17.644944 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.3:0
--local_ip_for_outbound_sockets=127.13.109.3
--webserver_interface=127.13.109.3
--webserver_port=0
--tserver_master_addrs=127.13.109.62:46061
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.3.keytab
--principal=kudu/127.13.109.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:43747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:00:17.767920 18520 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:00:17.768260 18520 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:00:17.768343 18520 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:00:17.772547 18520 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:00:17.772650 18520 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:00:17.772753 18520 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.3
I20260811 19:00:17.778415 18520 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:43747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.3.keytab
--principal=kudu/127.13.109.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.109.3
--webserver_port=0
--tserver_master_addrs=127.13.109.62:46061
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18520
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.3
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:00:17.779695 18520 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:00:17.780887 18520 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:00:17.791716 18525 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:17.791716 18527 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:17.791931 18520 server_base.cc:1034] running on GCE node
W20260811 19:00:17.791782 18529 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:17.792649 18520 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:00:17.793378 18520 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:00:17.794557 18520 hybrid_clock.cc:648] HybridClock initialized: now 1786474817794512 us; error 58 us; skew 500 ppm
Aug 11 19:00:17 dist-test-slave-0kls krb5kdc[18129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474817, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:00:17.797542 18520 init.cc:377] Logged in from keytab as kudu/127.13.109.3@KRBTEST.COM (short username kudu)
I20260811 19:00:17.798431 18520 fs_manager.cc:359] Metadata directory not provided
I20260811 19:00:17.798566 18520 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:00:17.798825 18520 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:00:17.801064 18520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/minicluster-data/ts-2/data/instance:
uuid: "41b6b533eb4042cea9f72da41e6b296b"
format_stamp: "Formatted at 2026-08-11 19:00:17 on dist-test-slave-0kls"
server_key: "45afecc96ef142a501659792f143e912"
server_key_iv: "53bf7f597d338322c271c4c78674cbe2"
server_key_version: "encryptionkey@0"
I20260811 19:00:17.801690 18520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/minicluster-data/ts-2/wal/instance:
uuid: "41b6b533eb4042cea9f72da41e6b296b"
format_stamp: "Formatted at 2026-08-11 19:00:17 on dist-test-slave-0kls"
server_key: "45afecc96ef142a501659792f143e912"
server_key_iv: "53bf7f597d338322c271c4c78674cbe2"
server_key_version: "encryptionkey@0"
I20260811 19:00:17.805711 18520 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260811 19:00:17.808506 18537 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:17.810127 18520 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 19:00:17.810307 18520 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/minicluster-data/ts-2/wal
uuid: "41b6b533eb4042cea9f72da41e6b296b"
format_stamp: "Formatted at 2026-08-11 19:00:17 on dist-test-slave-0kls"
server_key: "45afecc96ef142a501659792f143e912"
server_key_iv: "53bf7f597d338322c271c4c78674cbe2"
server_key_version: "encryptionkey@0"
I20260811 19:00:17.810456 18520 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:00:17.833511 18520 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:00:17.837020 18520 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:00:17.837276 18520 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:00:17.838047 18520 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:00:17.839015 18520 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:00:17.839094 18520 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:17.839167 18520 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:00:17.839226 18520 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:17.849296 18520 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.3:35221
I20260811 19:00:17.849323 18657 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.3:35221 every 8 connection(s)
I20260811 19:00:17.849973 18520 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
I20260811 19:00:17.853490 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 18520
I20260811 19:00:17.853644 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/minicluster-data/ts-2/wal/instance
I20260811 19:00:17.853984 13748 external_mini_cluster.cc:1428] Setting key 6f85c6e344db688f2b4fbdb8db69c338
Aug 11 19:00:17 dist-test-slave-0kls krb5kdc[18129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474817, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.3@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
I20260811 19:00:17.864696 18215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:17.851865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.3:35865 (local address 127.13.109.62:46061)
0811 19:00:17.852019 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:17.852023 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:17.852877 (+ 854us) server_negotiation.cc:407] Connection header received
0811 19:00:17.854655 (+ 1778us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:17.854659 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:17.854730 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:17.854824 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:17.856418 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:17.857004 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:17.858246 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:17.858464 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:17.858550 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:17.861418 (+ 2868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:17.861423 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:17.861425 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:17.861455 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:17.862789 (+ 1334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:17.863294 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:17.863296 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:17.863298 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:17.863377 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:17.863679 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:17.863682 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:17.863683 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:17.863871 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:17.863950 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:17.864452 (+ 502us) server_negotiation.cc:299] Negotiation successful
0811 19:00:17.864564 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 19:00:17.865581 18660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:17.852232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:46061 (local address 127.13.109.3:35865)
0811 19:00:17.852721 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:17.852768 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:17.854393 (+ 1625us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:17.855040 (+ 647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:17.855079 (+ 39us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:17.855610 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:17.856254 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:17.856278 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:17.857118 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:17.857120 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:17.858123 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:17.858133 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:17.858601 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:17.858603 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:17.858762 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:17.859339 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:17.859356 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:17.861191 (+ 1835us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:00:17.862905 (+ 1714us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:17.862910 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:17.862924 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:17.863204 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:17.863456 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:17.863458 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:17.863460 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:17.863598 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:17.863961 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:00:17.863966 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:00:17.864237 (+ 271us) client_negotiation.cc:769] Sending connection context
0811 19:00:17.864403 (+ 166us) client_negotiation.cc:240] Negotiation successful
0811 19:00:17.864678 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":124,"threads_started":1}
I20260811 19:00:17.866880 18658 heartbeater.cc:347] Connected to a master server at 127.13.109.62:46061
I20260811 19:00:17.867169 18658 heartbeater.cc:464] Registering TS with master...
I20260811 19:00:17.867830 18658 heartbeater.cc:511] Master 127.13.109.62:46061 requested a full tablet report, sending...
I20260811 19:00:17.868939 18174 ts_manager.cc:194] Registered new tserver with Master: 41b6b533eb4042cea9f72da41e6b296b (127.13.109.3:35221)
I20260811 19:00:17.869535 18174 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.109.3@KRBTEST.COM'} at 127.13.109.3:35865
I20260811 19:00:17.880330 13748 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 19:00:17.891000 18215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:17.882542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55810 (local address 127.13.109.62:46061)
0811 19:00:17.882721 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:17.882726 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:17.882885 (+ 159us) server_negotiation.cc:407] Connection header received
0811 19:00:17.883043 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:17.883047 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:17.883098 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:17.883190 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:17.884010 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:17.884658 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:17.885625 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:17.885944 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:17.886055 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:17.887586 (+ 1531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:17.887589 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:17.887592 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:17.887632 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:17.889036 (+ 1404us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:17.889510 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:17.889515 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:17.889516 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:17.889613 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:17.889993 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:17.889997 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:17.889999 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:17.890273 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:17.890428 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:17.890688 (+ 260us) server_negotiation.cc:299] Negotiation successful
0811 19:00:17.890830 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 19:00:17.894655 18174 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55810:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 19:00:17.898111 18174 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 19:00:17.913385 18671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:17.907606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.1:43603 (local address 127.0.0.1:43374)
0811 19:00:17.908060 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:17.908108 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:17.908244 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:17.908907 (+ 663us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:17.908914 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:17.908941 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:00:17.909125 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:17.909135 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:17.910626 (+ 1491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:17.910633 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:17.911685 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:17.911695 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:17.912782 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:17.912786 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:17.912904 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:17.912949 (+ 45us) client_negotiation.cc:769] Sending connection context
0811 19:00:17.913071 (+ 122us) client_negotiation.cc:240] Negotiation successful
0811 19:00:17.913217 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":110,"threads_started":1}
I20260811 19:00:17.913398 18672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:17.907758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43374 (local address 127.13.109.1:43603)
0811 19:00:17.908148 (+ 390us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:17.908156 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:17.908184 (+ 28us) server_negotiation.cc:407] Connection header received
0811 19:00:17.908453 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:17.908461 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:17.908721 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:17.909302 (+ 581us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:00:17.909454 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:17.910496 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:17.911823 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:17.912668 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:17.912793 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:17.912980 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:17.913115 (+ 135us) server_negotiation.cc:299] Negotiation successful
0811 19:00:17.913232 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":90,"threads_started":1}
I20260811 19:00:17.914417 18674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:17.908495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42732 (local address 127.13.109.3:35221)
0811 19:00:17.909235 (+ 740us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:17.909249 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:17.909275 (+ 26us) server_negotiation.cc:407] Connection header received
0811 19:00:17.909354 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:17.909359 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:17.909557 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:17.909728 (+ 171us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:00:17.910116 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:17.911306 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:17.912671 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:17.913516 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:17.913642 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:17.913830 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:17.914145 (+ 315us) server_negotiation.cc:299] Negotiation successful
0811 19:00:17.914253 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":637,"thread_start_us":157,"threads_started":1}
I20260811 19:00:17.914407 18673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:17.907655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.3:35221 (local address 127.0.0.1:42732)
0811 19:00:17.908740 (+ 1085us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:17.908755 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:17.908850 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:17.909817 (+ 967us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:17.909823 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:17.909855 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:00:17.909972 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:17.909979 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:17.911474 (+ 1495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:17.911477 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:17.912510 (+ 1033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:17.912522 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:17.913833 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:17.913837 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:17.913936 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:17.913996 (+ 60us) client_negotiation.cc:769] Sending connection context
0811 19:00:17.914088 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 19:00:17.914242 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1021,"mutex_wait_us":3,"spinlock_wait_cycles":1280,"thread_start_us":120,"threads_started":1}
I20260811 19:00:17.914412 18670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:17.908434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.2:45071 (local address 127.0.0.1:57580)
0811 19:00:17.908854 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:17.908868 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:17.908964 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:17.909814 (+ 850us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:17.909817 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:17.909845 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:00:17.909984 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:17.909989 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:17.911578 (+ 1589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:17.911583 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:17.912744 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:17.912757 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:17.913899 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:17.913904 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:17.913999 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:17.914033 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 19:00:17.914090 (+ 57us) client_negotiation.cc:240] Negotiation successful
0811 19:00:17.914209 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":351,"mutex_wait_us":35,"spinlock_wait_cycles":6400,"thread_start_us":103,"threads_started":1}
I20260811 19:00:17.915304 18675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:17.908930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57580 (local address 127.13.109.2:45071)
0811 19:00:17.909386 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:17.909396 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:17.909416 (+ 20us) server_negotiation.cc:407] Connection header received
0811 19:00:17.909479 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:17.909483 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:17.909633 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:17.910184 (+ 551us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:00:17.910358 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:17.911439 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:17.912881 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:17.913709 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:17.914825 (+ 1116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:17.914986 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:17.915038 (+ 52us) server_negotiation.cc:299] Negotiation successful
0811 19:00:17.915135 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":133,"threads_started":1}
I20260811 19:00:17.917518 18306 tablet_service.cc:1467] Processing CreateTablet for tablet 4b7a275dc36345039542f45da990901e (DEFAULT_TABLE table=test-table [id=e4ab7fabddaa49e287c769e68d3b748d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:00:17.917913 18587 tablet_service.cc:1467] Processing CreateTablet for tablet 4b7a275dc36345039542f45da990901e (DEFAULT_TABLE table=test-table [id=e4ab7fabddaa49e287c769e68d3b748d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:00:17.918058 18445 tablet_service.cc:1467] Processing CreateTablet for tablet 4b7a275dc36345039542f45da990901e (DEFAULT_TABLE table=test-table [id=e4ab7fabddaa49e287c769e68d3b748d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:00:17.918876 18306 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b7a275dc36345039542f45da990901e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:00:17.919009 18587 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b7a275dc36345039542f45da990901e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:00:17.919063 18445 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b7a275dc36345039542f45da990901e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:00:17.926021 18676 tablet_bootstrap.cc:492] T 4b7a275dc36345039542f45da990901e P 41b6b533eb4042cea9f72da41e6b296b: Bootstrap starting.
I20260811 19:00:17.927295 18677 tablet_bootstrap.cc:492] T 4b7a275dc36345039542f45da990901e P d7924d051943430495203c2582978852: Bootstrap starting.
I20260811 19:00:17.927743 18679 tablet_bootstrap.cc:492] T 4b7a275dc36345039542f45da990901e P 97d793a87a784870b326af48e8e6fe9b: Bootstrap starting.
I20260811 19:00:17.928678 18676 tablet_bootstrap.cc:654] T 4b7a275dc36345039542f45da990901e P 41b6b533eb4042cea9f72da41e6b296b: Neither blocks nor log segments found. Creating new log.
I20260811 19:00:17.929318 18677 tablet_bootstrap.cc:654] T 4b7a275dc36345039542f45da990901e P d7924d051943430495203c2582978852: Neither blocks nor log segments found. Creating new log.
I20260811 19:00:17.930143 18676 log.cc:824] T 4b7a275dc36345039542f45da990901e P 41b6b533eb4042cea9f72da41e6b296b: Log is configured to *not* fsync() on all Append() calls
I20260811 19:00:17.930207 18677 log.cc:824] T 4b7a275dc36345039542f45da990901e P d7924d051943430495203c2582978852: Log is configured to *not* fsync() on all Append() calls
I20260811 19:00:17.930516 18679 tablet_bootstrap.cc:654] T 4b7a275dc36345039542f45da990901e P 97d793a87a784870b326af48e8e6fe9b: Neither blocks nor log segments found. Creating new log.
I20260811 19:00:17.931555 18679 log.cc:824] T 4b7a275dc36345039542f45da990901e P 97d793a87a784870b326af48e8e6fe9b: Log is configured to *not* fsync() on all Append() calls
I20260811 19:00:17.932617 18677 tablet_bootstrap.cc:492] T 4b7a275dc36345039542f45da990901e P d7924d051943430495203c2582978852: No bootstrap required, opened a new log
I20260811 19:00:17.932701 18676 tablet_bootstrap.cc:492] T 4b7a275dc36345039542f45da990901e P 41b6b533eb4042cea9f72da41e6b296b: No bootstrap required, opened a new log
I20260811 19:00:17.932859 18677 ts_tablet_manager.cc:1397] T 4b7a275dc36345039542f45da990901e P d7924d051943430495203c2582978852: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 19:00:17.932896 18676 ts_tablet_manager.cc:1397] T 4b7a275dc36345039542f45da990901e P 41b6b533eb4042cea9f72da41e6b296b: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 19:00:17.933691 18679 tablet_bootstrap.cc:492] T 4b7a275dc36345039542f45da990901e P 97d793a87a784870b326af48e8e6fe9b: No bootstrap required, opened a new log
I20260811 19:00:17.933924 18679 ts_tablet_manager.cc:1397] T 4b7a275dc36345039542f45da990901e P 97d793a87a784870b326af48e8e6fe9b: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20260811 19:00:17.936504 18677 raft_consensus.cc:348] T 4b7a275dc36345039542f45da990901e P d7924d051943430495203c2582978852 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7924d051943430495203c2582978852" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 43603 } } peers { permanent_uuid: "41b6b533eb4042cea9f72da41e6b296b" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 35221 } } peers { permanent_uuid: "97d793a87a784870b326af48e8e6fe9b" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 45071 } }
I20260811 19:00:17.936810 18677 raft_consensus.cc:374] T 4b7a275dc36345039542f45da990901e P d7924d051943430495203c2582978852 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:00:17.936877 18677 raft_consensus.cc:729] T 4b7a275dc36345039542f45da990901e P d7924d051943430495203c2582978852 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7924d051943430495203c2582978852, State: Initialized, Role: FOLLOWER
I20260811 19:00:17.936959 18676 raft_consensus.cc:348] T 4b7a275dc36345039542f45da990901e P 41b6b533eb4042cea9f72da41e6b296b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7924d051943430495203c2582978852" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 43603 } } peers { permanent_uuid: "41b6b533eb4042cea9f72da41e6b296b" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 35221 } } peers { permanent_uuid: "97d793a87a784870b326af48e8e6fe9b" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 45071 } }
I20260811 19:00:17.937256 18676 raft_consensus.cc:374] T 4b7a275dc36345039542f45da990901e P 41b6b533eb4042cea9f72da41e6b296b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:00:17.937319 18676 raft_consensus.cc:729] T 4b7a275dc36345039542f45da990901e P 41b6b533eb4042cea9f72da41e6b296b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41b6b533eb4042cea9f72da41e6b296b, State: Initialized, Role: FOLLOWER
I20260811 19:00:17.937409 18677 consensus_queue.cc:260] T 4b7a275dc36345039542f45da990901e P d7924d051943430495203c2582978852 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7924d051943430495203c2582978852" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 43603 } } peers { permanent_uuid: "41b6b533eb4042cea9f72da41e6b296b" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 35221 } } peers { permanent_uuid: "97d793a87a784870b326af48e8e6fe9b" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 45071 } }
I20260811 19:00:17.937749 18676 consensus_queue.cc:260] T 4b7a275dc36345039542f45da990901e P 41b6b533eb4042cea9f72da41e6b296b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7924d051943430495203c2582978852" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 43603 } } peers { permanent_uuid: "41b6b533eb4042cea9f72da41e6b296b" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 35221 } } peers { permanent_uuid: "97d793a87a784870b326af48e8e6fe9b" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 45071 } }
I20260811 19:00:17.937885 18679 raft_consensus.cc:348] T 4b7a275dc36345039542f45da990901e P 97d793a87a784870b326af48e8e6fe9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7924d051943430495203c2582978852" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 43603 } } peers { permanent_uuid: "41b6b533eb4042cea9f72da41e6b296b" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 35221 } } peers { permanent_uuid: "97d793a87a784870b326af48e8e6fe9b" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 45071 } }
I20260811 19:00:17.938149 18679 raft_consensus.cc:374] T 4b7a275dc36345039542f45da990901e P 97d793a87a784870b326af48e8e6fe9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:00:17.938203 18679 raft_consensus.cc:729] T 4b7a275dc36345039542f45da990901e P 97d793a87a784870b326af48e8e6fe9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97d793a87a784870b326af48e8e6fe9b, State: Initialized, Role: FOLLOWER
I20260811 19:00:17.938633 18677 ts_tablet_manager.cc:1428] T 4b7a275dc36345039542f45da990901e P d7924d051943430495203c2582978852: Time spent starting tablet: real 0.006s user 0.002s sys 0.003s
I20260811 19:00:17.938658 18679 consensus_queue.cc:260] T 4b7a275dc36345039542f45da990901e P 97d793a87a784870b326af48e8e6fe9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7924d051943430495203c2582978852" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 43603 } } peers { permanent_uuid: "41b6b533eb4042cea9f72da41e6b296b" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 35221 } } peers { permanent_uuid: "97d793a87a784870b326af48e8e6fe9b" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 45071 } }
I20260811 19:00:17.938809 18676 ts_tablet_manager.cc:1428] T 4b7a275dc36345039542f45da990901e P 41b6b533eb4042cea9f72da41e6b296b: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 19:00:17.939029 18376 heartbeater.cc:503] Master 127.13.109.62:46061 was elected leader, sending a full tablet report...
I20260811 19:00:17.939159 18658 heartbeater.cc:503] Master 127.13.109.62:46061 was elected leader, sending a full tablet report...
I20260811 19:00:17.939576 18679 ts_tablet_manager.cc:1428] T 4b7a275dc36345039542f45da990901e P 97d793a87a784870b326af48e8e6fe9b: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260811 19:00:17.939965 18514 heartbeater.cc:503] Master 127.13.109.62:46061 was elected leader, sending a full tablet report...
W20260811 19:00:18.100944 18659 tablet.cc:2369] T 4b7a275dc36345039542f45da990901e P 41b6b533eb4042cea9f72da41e6b296b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 19:00:18.104736 18516 tablet.cc:2369] T 4b7a275dc36345039542f45da990901e P 97d793a87a784870b326af48e8e6fe9b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 19:00:18.105437 18377 tablet.cc:2369] T 4b7a275dc36345039542f45da990901e P d7924d051943430495203c2582978852: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 19:00:18.225452 18683 raft_consensus.cc:482] T 4b7a275dc36345039542f45da990901e P d7924d051943430495203c2582978852 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:00:18.225741 18683 raft_consensus.cc:504] T 4b7a275dc36345039542f45da990901e P d7924d051943430495203c2582978852 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7924d051943430495203c2582978852" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 43603 } } peers { permanent_uuid: "41b6b533eb4042cea9f72da41e6b296b" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 35221 } } peers { permanent_uuid: "97d793a87a784870b326af48e8e6fe9b" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 45071 } }
I20260811 19:00:18.227089 18683 leader_election.cc:288] T 4b7a275dc36345039542f45da990901e P d7924d051943430495203c2582978852 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41b6b533eb4042cea9f72da41e6b296b (127.13.109.3:35221), 97d793a87a784870b326af48e8e6fe9b (127.13.109.2:45071)
I20260811 19:00:18.231803 18675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:18.227517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.1:41495 (local address 127.13.109.2:45071)
0811 19:00:18.227660 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:18.227665 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:18.227956 (+ 291us) server_negotiation.cc:407] Connection header received
0811 19:00:18.228062 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:18.228068 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:18.228131 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:18.228216 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:00:18.228580 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:18.229251 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:18.230741 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:18.231145 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:18.231252 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:18.231337 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:18.231570 (+ 233us) server_negotiation.cc:299] Negotiation successful
0811 19:00:18.231654 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 19:00:18.231825 18674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:18.227521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.1:53829 (local address 127.13.109.3:35221)
0811 19:00:18.227687 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:18.227693 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:18.228131 (+ 438us) server_negotiation.cc:407] Connection header received
0811 19:00:18.228260 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:18.228263 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:18.228318 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:18.228405 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:00:18.228749 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:18.229435 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:18.230774 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:18.231178 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:18.231331 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:18.231392 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:18.231623 (+ 231us) server_negotiation.cc:299] Negotiation successful
0811 19:00:18.231711 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 19:00:18.231828 18687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:18.227415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.3:35221 (local address 127.13.109.1:53829)
0811 19:00:18.228014 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:18.228030 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:18.228137 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:18.228425 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:18.228428 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:18.228443 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:00:18.228578 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:18.228585 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:18.229565 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:18.229568 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:18.230617 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:18.230628 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:18.231358 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:18.231364 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:18.231458 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:18.231477 (+ 19us) client_negotiation.cc:769] Sending connection context
0811 19:00:18.231580 (+ 103us) client_negotiation.cc:240] Negotiation successful
0811 19:00:18.231653 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":517,"mutex_wait_us":67,"thread_start_us":126,"threads_started":1}
I20260811 19:00:18.231840 18686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:18.227414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.2:45071 (local address 127.13.109.1:41495)
0811 19:00:18.227834 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:18.227846 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:18.227959 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:18.228249 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:18.228252 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:18.228273 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:00:18.228442 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:18.228450 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:18.229412 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:18.229415 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:18.230593 (+ 1178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:18.230604 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:18.231290 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:18.231294 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:18.231418 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:18.231451 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 19:00:18.231526 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 19:00:18.231639 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":360,"mutex_wait_us":41,"thread_start_us":117,"threads_started":1}
I20260811 19:00:18.232793 18607 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b7a275dc36345039542f45da990901e" candidate_uuid: "d7924d051943430495203c2582978852" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41b6b533eb4042cea9f72da41e6b296b" is_pre_election: true
I20260811 19:00:18.232991 18466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b7a275dc36345039542f45da990901e" candidate_uuid: "d7924d051943430495203c2582978852" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97d793a87a784870b326af48e8e6fe9b" is_pre_election: true
I20260811 19:00:18.233188 18607 raft_consensus.cc:2454] T 4b7a275dc36345039542f45da990901e P 41b6b533eb4042cea9f72da41e6b296b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7924d051943430495203c2582978852 in term 0.
I20260811 19:00:18.233345 18466 raft_consensus.cc:2454] T 4b7a275dc36345039542f45da990901e P 97d793a87a784870b326af48e8e6fe9b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7924d051943430495203c2582978852 in term 0.
I20260811 19:00:18.233744 18259 leader_election.cc:302] T 4b7a275dc36345039542f45da990901e P d7924d051943430495203c2582978852 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41b6b533eb4042cea9f72da41e6b296b, d7924d051943430495203c2582978852; no voters:
I20260811 19:00:18.234117 18683 raft_consensus.cc:2781] T 4b7a275dc36345039542f45da990901e P d7924d051943430495203c2582978852 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 19:00:18.234215 18683 raft_consensus.cc:482] T 4b7a275dc36345039542f45da990901e P d7924d051943430495203c2582978852 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 19:00:18.234266 18683 raft_consensus.cc:3037] T 4b7a275dc36345039542f45da990901e P d7924d051943430495203c2582978852 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:00:18.235406 18683 raft_consensus.cc:504] T 4b7a275dc36345039542f45da990901e P d7924d051943430495203c2582978852 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7924d051943430495203c2582978852" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 43603 } } peers { permanent_uuid: "41b6b533eb4042cea9f72da41e6b296b" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 35221 } } peers { permanent_uuid: "97d793a87a784870b326af48e8e6fe9b" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 45071 } }
I20260811 19:00:18.235838 18683 leader_election.cc:288] T 4b7a275dc36345039542f45da990901e P d7924d051943430495203c2582978852 [CANDIDATE]: Term 1 election: Requested vote from peers 41b6b533eb4042cea9f72da41e6b296b (127.13.109.3:35221), 97d793a87a784870b326af48e8e6fe9b (127.13.109.2:45071)
I20260811 19:00:18.236312 18466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b7a275dc36345039542f45da990901e" candidate_uuid: "d7924d051943430495203c2582978852" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97d793a87a784870b326af48e8e6fe9b"
I20260811 19:00:18.236318 18607 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b7a275dc36345039542f45da990901e" candidate_uuid: "d7924d051943430495203c2582978852" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41b6b533eb4042cea9f72da41e6b296b"
I20260811 19:00:18.236431 18466 raft_consensus.cc:3037] T 4b7a275dc36345039542f45da990901e P 97d793a87a784870b326af48e8e6fe9b [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:00:18.236469 18607 raft_consensus.cc:3037] T 4b7a275dc36345039542f45da990901e P 41b6b533eb4042cea9f72da41e6b296b [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:00:18.237591 18466 raft_consensus.cc:2454] T 4b7a275dc36345039542f45da990901e P 97d793a87a784870b326af48e8e6fe9b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7924d051943430495203c2582978852 in term 1.
I20260811 19:00:18.237682 18607 raft_consensus.cc:2454] T 4b7a275dc36345039542f45da990901e P 41b6b533eb4042cea9f72da41e6b296b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7924d051943430495203c2582978852 in term 1.
I20260811 19:00:18.237994 18260 leader_election.cc:302] T 4b7a275dc36345039542f45da990901e P d7924d051943430495203c2582978852 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 97d793a87a784870b326af48e8e6fe9b, d7924d051943430495203c2582978852; no voters:
I20260811 19:00:18.238237 18683 raft_consensus.cc:2781] T 4b7a275dc36345039542f45da990901e P d7924d051943430495203c2582978852 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:00:18.238420 18683 raft_consensus.cc:686] T 4b7a275dc36345039542f45da990901e P d7924d051943430495203c2582978852 [term 1 LEADER]: Becoming Leader. State: Replica: d7924d051943430495203c2582978852, State: Running, Role: LEADER
I20260811 19:00:18.238782 18683 consensus_queue.cc:237] T 4b7a275dc36345039542f45da990901e P d7924d051943430495203c2582978852 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7924d051943430495203c2582978852" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 43603 } } peers { permanent_uuid: "41b6b533eb4042cea9f72da41e6b296b" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 35221 } } peers { permanent_uuid: "97d793a87a784870b326af48e8e6fe9b" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 45071 } }
I20260811 19:00:18.242429 18173 catalog_manager.cc:5495] T 4b7a275dc36345039542f45da990901e P d7924d051943430495203c2582978852 reported cstate change: term changed from 0 to 1, leader changed from <none> to d7924d051943430495203c2582978852 (127.13.109.1). New cstate: current_term: 1 leader_uuid: "d7924d051943430495203c2582978852" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7924d051943430495203c2582978852" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41b6b533eb4042cea9f72da41e6b296b" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 35221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97d793a87a784870b326af48e8e6fe9b" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 45071 } health_report { overall_health: UNKNOWN } } }
I20260811 19:00:18.282454 18672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:18.278232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43384 (local address 127.13.109.1:43603)
0811 19:00:18.278385 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:18.278390 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:18.278408 (+ 18us) server_negotiation.cc:407] Connection header received
0811 19:00:18.278578 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:18.278581 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:18.278644 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:18.278771 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:00:18.279093 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:18.279642 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:18.280642 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:18.280946 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:18.281078 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:18.281446 (+ 368us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:00:18.282001 (+ 555us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:00:18.282151 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:18.282211 (+ 60us) server_negotiation.cc:299] Negotiation successful
0811 19:00:18.282279 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 19:00:18.289319 18607 raft_consensus.cc:1261] T 4b7a275dc36345039542f45da990901e P 41b6b533eb4042cea9f72da41e6b296b [term 1 FOLLOWER]: Refusing update from remote peer d7924d051943430495203c2582978852: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:00:18.289341 18466 raft_consensus.cc:1261] T 4b7a275dc36345039542f45da990901e P 97d793a87a784870b326af48e8e6fe9b [term 1 FOLLOWER]: Refusing update from remote peer d7924d051943430495203c2582978852: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:00:18.290207 18683 consensus_queue.cc:1035] T 4b7a275dc36345039542f45da990901e P d7924d051943430495203c2582978852 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97d793a87a784870b326af48e8e6fe9b" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 45071 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:00:18.290525 18688 consensus_queue.cc:1035] T 4b7a275dc36345039542f45da990901e P d7924d051943430495203c2582978852 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41b6b533eb4042cea9f72da41e6b296b" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 35221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:00:18.299454 18693 mvcc.cc:205] Tried to move back new op lower bound from 7317400855704612864 to 7317400855508942848. Current Snapshot: MvccSnapshot[applied={T|T < 7317400855704612864}]
I20260811 19:00:18.308511 18173 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55810:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:00:18.308737 18173 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55810:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:00:18.311513 18173 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e1102f5c3f804fc7947b5b44dd58c9b4: Sending DeleteTablet for 3 replicas of tablet 4b7a275dc36345039542f45da990901e
I20260811 19:00:18.312518 18587 tablet_service.cc:1514] Processing DeleteTablet for tablet 4b7a275dc36345039542f45da990901e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:00:18 UTC) from {username='kudu', principal='kudu/127.13.109.62@KRBTEST.COM'} at 127.0.0.1:42732
I20260811 19:00:18.312573 18445 tablet_service.cc:1514] Processing DeleteTablet for tablet 4b7a275dc36345039542f45da990901e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:00:18 UTC) from {username='kudu', principal='kudu/127.13.109.62@KRBTEST.COM'} at 127.0.0.1:57580
I20260811 19:00:18.312733 18306 tablet_service.cc:1514] Processing DeleteTablet for tablet 4b7a275dc36345039542f45da990901e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:00:18 UTC) from {username='kudu', principal='kudu/127.13.109.62@KRBTEST.COM'} at 127.0.0.1:43374
I20260811 19:00:18.313126 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 18240
I20260811 19:00:18.316879 18702 tablet_replica.cc:330] T 4b7a275dc36345039542f45da990901e P 97d793a87a784870b326af48e8e6fe9b: stopping tablet replica
I20260811 19:00:18.317340 18702 raft_consensus.cc:2229] T 4b7a275dc36345039542f45da990901e P 97d793a87a784870b326af48e8e6fe9b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:00:18.317726 18702 raft_consensus.cc:2258] T 4b7a275dc36345039542f45da990901e P 97d793a87a784870b326af48e8e6fe9b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:00:18.320371 18702 ts_tablet_manager.cc:1905] T 4b7a275dc36345039542f45da990901e P 97d793a87a784870b326af48e8e6fe9b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:00:18.320871 18701 tablet_replica.cc:330] T 4b7a275dc36345039542f45da990901e P 41b6b533eb4042cea9f72da41e6b296b: stopping tablet replica
I20260811 19:00:18.321290 18701 raft_consensus.cc:2229] T 4b7a275dc36345039542f45da990901e P 41b6b533eb4042cea9f72da41e6b296b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:00:18.321684 18701 raft_consensus.cc:2258] T 4b7a275dc36345039542f45da990901e P 41b6b533eb4042cea9f72da41e6b296b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:00:18.323684 18702 ts_tablet_manager.cc:1918] T 4b7a275dc36345039542f45da990901e P 97d793a87a784870b326af48e8e6fe9b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:00:18.323800 18702 log.cc:1196] T 4b7a275dc36345039542f45da990901e P 97d793a87a784870b326af48e8e6fe9b: Deleting WAL directory at /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/minicluster-data/ts-1/wal/wals/4b7a275dc36345039542f45da990901e
I20260811 19:00:18.324159 18702 ts_tablet_manager.cc:1939] T 4b7a275dc36345039542f45da990901e P 97d793a87a784870b326af48e8e6fe9b: Deleting consensus metadata
I20260811 19:00:18.325206 18701 ts_tablet_manager.cc:1905] T 4b7a275dc36345039542f45da990901e P 41b6b533eb4042cea9f72da41e6b296b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:00:18.325568 18159 catalog_manager.cc:4841] TS 97d793a87a784870b326af48e8e6fe9b (127.13.109.2:45071): tablet 4b7a275dc36345039542f45da990901e (table test-table [id=e4ab7fabddaa49e287c769e68d3b748d]) successfully deleted
I20260811 19:00:18.327889 18701 ts_tablet_manager.cc:1918] T 4b7a275dc36345039542f45da990901e P 41b6b533eb4042cea9f72da41e6b296b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:00:18.327980 18701 log.cc:1196] T 4b7a275dc36345039542f45da990901e P 41b6b533eb4042cea9f72da41e6b296b: Deleting WAL directory at /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474792496492-13748-0/minicluster-data/ts-2/wal/wals/4b7a275dc36345039542f45da990901e
I20260811 19:00:18.328290 18701 ts_tablet_manager.cc:1939] T 4b7a275dc36345039542f45da990901e P 41b6b533eb4042cea9f72da41e6b296b: Deleting consensus metadata
I20260811 19:00:18.329754 18158 catalog_manager.cc:4841] TS 41b6b533eb4042cea9f72da41e6b296b (127.13.109.3:35221): tablet 4b7a275dc36345039542f45da990901e (table test-table [id=e4ab7fabddaa49e287c769e68d3b748d]) successfully deleted
W20260811 19:00:18.331025 18159 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.109.1:43603 (error 108)
I20260811 19:00:18.332105 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 18381
I20260811 19:00:18.332520 18670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:18.332127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.1:43603 (local address 127.0.0.1:43386)
0811 19:00:18.332312 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:18.332405 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.1:43603: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260811 19:00:18.333297 18159 catalog_manager.cc:4584] TS d7924d051943430495203c2582978852 (127.13.109.1:43603): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4b7a275dc36345039542f45da990901e: Network error: Client connection negotiation failed: client connection to 127.13.109.1:43603: connect: Connection refused (error 111)
I20260811 19:00:18.341226 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 18520
I20260811 19:00:18.350248 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 18142
2026-08-11T19:00:18Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2718 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-11T19:00:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:00:18Z Disabled control of system clock
I20260811 19:00:18.385507 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474792496492-13748-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474792496492-13748-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474792496492-13748-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.62:38883
--webserver_interface=127.13.109.62
--webserver_port=0
--builtin_ntp_servers=127.13.109.20:34935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.109.62:38883
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 19:00:18.508747 18709 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:00:18.509073 18709 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:00:18.509122 18709 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:00:18.513585 18709 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:00:18.513675 18709 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:00:18.513691 18709 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:00:18.513708 18709 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:00:18.513725 18709 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:00:18.519272 18709 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:34935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474792496492-13748-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474792496492-13748-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.109.62:38883
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.62:38883
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.109.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18709
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474792496492-13748-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:00:18.520478 18709 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:00:18.522082 18709 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:00:18.531097 18715 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:18.531109 18718 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:18.531324 18709 server_base.cc:1034] running on GCE node
W20260811 19:00:18.531126 18716 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:18.532142 18709 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:00:18.533417 18709 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:00:18.534610 18709 hybrid_clock.cc:648] HybridClock initialized: now 1786474818534568 us; error 63 us; skew 500 ppm
I20260811 19:00:18.536664 18709 webserver.cc:460] Webserver started at http://127.13.109.62:41355/ using document root <none> and password file <none>
I20260811 19:00:18.537298 18709 fs_manager.cc:359] Metadata directory not provided
I20260811 19:00:18.537398 18709 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:00:18.537652 18709 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:00:18.539986 18709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474792496492-13748-0/minicluster-data/master-0/data/instance:
uuid: "13c92f3a1083414a9cc49f4604b90270"
format_stamp: "Formatted at 2026-08-11 19:00:18 on dist-test-slave-0kls"
server_key: "1eb3d2bdaa32c56004ffcd60d342fd13"
server_key_iv: "340bc6ac0c829122b1a40af83e35fb7d"
server_key_version: "encryptionkey@0"
I20260811 19:00:18.540611 18709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474792496492-13748-0/minicluster-data/master-0/wal/instance:
uuid: "13c92f3a1083414a9cc49f4604b90270"
format_stamp: "Formatted at 2026-08-11 19:00:18 on dist-test-slave-0kls"
server_key: "1eb3d2bdaa32c56004ffcd60d342fd13"
server_key_iv: "340bc6ac0c829122b1a40af83e35fb7d"
server_key_version: "encryptionkey@0"
I20260811 19:00:18.544647 18709 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 19:00:18.547397 18725 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:18.548717 18709 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 19:00:18.548877 18709 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474792496492-13748-0/minicluster-data/master-0/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474792496492-13748-0/minicluster-data/master-0/wal
uuid: "13c92f3a1083414a9cc49f4604b90270"
format_stamp: "Formatted at 2026-08-11 19:00:18 on dist-test-slave-0kls"
server_key: "1eb3d2bdaa32c56004ffcd60d342fd13"
server_key_iv: "340bc6ac0c829122b1a40af83e35fb7d"
server_key_version: "encryptionkey@0"
I20260811 19:00:18.549009 18709 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474792496492-13748-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474792496492-13748-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474792496492-13748-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:00:18.554072 18709 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:00:18.554750 18709 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:00:18.554951 18709 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:00:18.562248 18781 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.62:38883 every 8 connection(s)
I20260811 19:00:18.562249 18709 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.62:38883
I20260811 19:00:18.563313 18709 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
I20260811 19:00:18.566746 18782 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:00:18.572290 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 18709
I20260811 19:00:18.572425 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474792496492-13748-0/minicluster-data/master-0/wal/instance
I20260811 19:00:18.572796 13748 external_mini_cluster.cc:1428] Setting key 3499f8978018ef4a2ed5e74af968d739
I20260811 19:00:18.573489 18782 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13c92f3a1083414a9cc49f4604b90270: Bootstrap starting.
I20260811 19:00:18.576522 18782 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 13c92f3a1083414a9cc49f4604b90270: Neither blocks nor log segments found. Creating new log.
I20260811 19:00:18.577499 18782 log.cc:824] T 00000000000000000000000000000000 P 13c92f3a1083414a9cc49f4604b90270: Log is configured to *not* fsync() on all Append() calls
I20260811 19:00:18.579422 18785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:18.574427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37554 (local address 127.13.109.62:38883)
0811 19:00:18.575313 (+ 886us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:18.575334 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:18.575384 (+ 50us) server_negotiation.cc:407] Connection header received
0811 19:00:18.576308 (+ 924us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:18.576335 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:18.576845 (+ 510us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:18.577239 (+ 394us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:18.578005 (+ 766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:18.578013 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:18.578030 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:00:18.578042 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:18.578107 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:00:18.578123 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:18.578225 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:18.578387 (+ 162us) server_negotiation.cc:299] Negotiation successful
0811 19:00:18.578685 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":140,"threads_started":1}
I20260811 19:00:18.579965 18782 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13c92f3a1083414a9cc49f4604b90270: No bootstrap required, opened a new log
I20260811 19:00:18.583987 18782 raft_consensus.cc:348] T 00000000000000000000000000000000 P 13c92f3a1083414a9cc49f4604b90270 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13c92f3a1083414a9cc49f4604b90270" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 38883 } }
I20260811 19:00:18.584277 18782 raft_consensus.cc:374] T 00000000000000000000000000000000 P 13c92f3a1083414a9cc49f4604b90270 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:00:18.584316 18782 raft_consensus.cc:729] T 00000000000000000000000000000000 P 13c92f3a1083414a9cc49f4604b90270 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13c92f3a1083414a9cc49f4604b90270, State: Initialized, Role: FOLLOWER
I20260811 19:00:18.584785 18782 consensus_queue.cc:260] T 00000000000000000000000000000000 P 13c92f3a1083414a9cc49f4604b90270 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13c92f3a1083414a9cc49f4604b90270" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 38883 } }
I20260811 19:00:18.584924 18782 raft_consensus.cc:388] T 00000000000000000000000000000000 P 13c92f3a1083414a9cc49f4604b90270 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:00:18.584976 18782 raft_consensus.cc:482] T 00000000000000000000000000000000 P 13c92f3a1083414a9cc49f4604b90270 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:00:18.585094 18782 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 13c92f3a1083414a9cc49f4604b90270 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:00:18.586380 18782 raft_consensus.cc:504] T 00000000000000000000000000000000 P 13c92f3a1083414a9cc49f4604b90270 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13c92f3a1083414a9cc49f4604b90270" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 38883 } }
I20260811 19:00:18.586774 18782 leader_election.cc:302] T 00000000000000000000000000000000 P 13c92f3a1083414a9cc49f4604b90270 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 13c92f3a1083414a9cc49f4604b90270; no voters:
I20260811 19:00:18.587175 18782 leader_election.cc:288] T 00000000000000000000000000000000 P 13c92f3a1083414a9cc49f4604b90270 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:00:18.587304 18787 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 13c92f3a1083414a9cc49f4604b90270 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:00:18.587571 18787 raft_consensus.cc:686] T 00000000000000000000000000000000 P 13c92f3a1083414a9cc49f4604b90270 [term 1 LEADER]: Becoming Leader. State: Replica: 13c92f3a1083414a9cc49f4604b90270, State: Running, Role: LEADER
I20260811 19:00:18.587953 18787 consensus_queue.cc:237] T 00000000000000000000000000000000 P 13c92f3a1083414a9cc49f4604b90270 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13c92f3a1083414a9cc49f4604b90270" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 38883 } }
I20260811 19:00:18.588898 18782 sys_catalog.cc:564] T 00000000000000000000000000000000 P 13c92f3a1083414a9cc49f4604b90270 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:00:18.590420 18789 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13c92f3a1083414a9cc49f4604b90270 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "13c92f3a1083414a9cc49f4604b90270" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13c92f3a1083414a9cc49f4604b90270" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 38883 } } }
I20260811 19:00:18.590651 18789 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13c92f3a1083414a9cc49f4604b90270 [sys.catalog]: This master's current role is: LEADER
I20260811 19:00:18.590981 18790 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13c92f3a1083414a9cc49f4604b90270 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 13c92f3a1083414a9cc49f4604b90270. Latest consensus state: current_term: 1 leader_uuid: "13c92f3a1083414a9cc49f4604b90270" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13c92f3a1083414a9cc49f4604b90270" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 38883 } } }
I20260811 19:00:18.591101 18790 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13c92f3a1083414a9cc49f4604b90270 [sys.catalog]: This master's current role is: LEADER
I20260811 19:00:18.591282 18797 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:00:18.594978 18797 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:00:18.601521 18797 catalog_manager.cc:1347] Generated new cluster ID: aa519ec33edb4bf4a8861d7fb73a74cb
I20260811 19:00:18.601616 18797 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:00:18.616529 18797 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:00:18.616905 18797 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:00:18.624972 18797 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 13c92f3a1083414a9cc49f4604b90270: Generated new TSK 0
I20260811 19:00:18.625766 18797 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 19:00:18.640864 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474792496492-13748-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474792496492-13748-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474792496492-13748-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.1:0
--local_ip_for_outbound_sockets=127.13.109.1
--webserver_interface=127.13.109.1
--webserver_port=0
--tserver_master_addrs=127.13.109.62:38883
--builtin_ntp_servers=127.13.109.20:34935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 19:00:18.902031 18807 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:00:18.902483 18807 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:00:18.902534 18807 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:00:18.907476 18807 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:00:18.907583 18807 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:00:18.907655 18807 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.1
I20260811 19:00:18.913542 18807 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:34935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474792496492-13748-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474792496492-13748-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.109.1
--webserver_port=0
--tserver_master_addrs=127.13.109.62:38883
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18807
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.1
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474792496492-13748-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:00:18.915371 18807 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:00:18.917371 18807 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:00:18.930007 18812 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:18.930012 18815 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:18.930101 18813 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:18.930402 18807 server_base.cc:1034] running on GCE node
I20260811 19:00:18.931120 18807 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:00:18.931959 18807 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:00:18.933157 18807 hybrid_clock.cc:648] HybridClock initialized: now 1786474818933134 us; error 33 us; skew 500 ppm
I20260811 19:00:18.935604 18807 webserver.cc:460] Webserver started at http://127.13.109.1:33249/ using document root <none> and password file <none>
I20260811 19:00:18.936337 18807 fs_manager.cc:359] Metadata directory not provided
I20260811 19:00:18.936453 18807 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:00:18.936712 18807 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:00:18.939347 18807 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474792496492-13748-0/minicluster-data/ts-0/data/instance:
uuid: "1ebaaf751ba34262ba390a70ddab9048"
format_stamp: "Formatted at 2026-08-11 19:00:18 on dist-test-slave-0kls"
server_key: "8dc344e7b8227262779151ff708bcf03"
server_key_iv: "4a8d348c04ffff95338034be40965138"
server_key_version: "encryptionkey@0"
I20260811 19:00:18.940062 18807 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474792496492-13748-0/minicluster-data/ts-0/wal/instance:
uuid: "1ebaaf751ba34262ba390a70ddab9048"
format_stamp: "Formatted at 2026-08-11 19:00:18 on dist-test-slave-0kls"
server_key: "8dc344e7b8227262779151ff708bcf03"
server_key_iv: "4a8d348c04ffff95338034be40965138"
server_key_version: "encryptionkey@0"
I20260811 19:00:18.944705 18807 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 19:00:18.947929 18822 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:18.949343 18807 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 19:00:18.949466 18807 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474792496492-13748-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474792496492-13748-0/minicluster-data/ts-0/wal
uuid: "1ebaaf751ba34262ba390a70ddab9048"
format_stamp: "Formatted at 2026-08-11 19:00:18 on dist-test-slave-0kls"
server_key: "8dc344e7b8227262779151ff708bcf03"
server_key_iv: "4a8d348c04ffff95338034be40965138"
server_key_version: "encryptionkey@0"
I20260811 19:00:18.949581 18807 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474792496492-13748-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474792496492-13748-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474792496492-13748-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:00:18.954869 18807 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:00:18.955531 18807 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:00:18.955693 18807 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:00:18.956285 18807 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:00:18.957350 18807 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:00:18.957401 18807 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:18.957444 18807 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:00:18.957459 18807 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:18.966893 18807 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.1:40505
I20260811 19:00:18.966936 18943 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.1:40505 every 8 connection(s)
I20260811 19:00:18.967945 18807 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
I20260811 19:00:18.974458 18785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:18.970054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.1:58119 (local address 127.13.109.62:38883)
0811 19:00:18.970213 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:18.970216 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:18.971369 (+ 1153us) server_negotiation.cc:407] Connection header received
0811 19:00:18.972812 (+ 1443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:18.972814 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:18.972857 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:18.972937 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:18.974101 (+ 1164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:18.974104 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:18.974107 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:00:18.974121 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:18.974141 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:00:18.974149 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:18.974233 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:18.974302 (+ 69us) server_negotiation.cc:299] Negotiation successful
0811 19:00:18.974341 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 19:00:18.975526 18946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:18.970503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:38883 (local address 127.13.109.1:58119)
0811 19:00:18.971228 (+ 725us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:18.971272 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:18.972600 (+ 1328us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:18.973127 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:18.973134 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:18.973233 (+ 99us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:18.973776 (+ 543us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 19:00:18.973778 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:18.973855 (+ 77us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 19:00:18.973930 (+ 75us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:00:18.974210 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:00:18.974214 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:00:18.974218 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 19:00:18.974346 (+ 128us) client_negotiation.cc:240] Negotiation successful
0811 19:00:18.974679 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":131,"threads_started":1}
I20260811 19:00:18.976881 18944 heartbeater.cc:347] Connected to a master server at 127.13.109.62:38883
I20260811 19:00:18.977164 18944 heartbeater.cc:464] Registering TS with master...
I20260811 19:00:18.977407 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 18807
I20260811 19:00:18.977499 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474792496492-13748-0/minicluster-data/ts-0/wal/instance
I20260811 19:00:18.977694 18944 heartbeater.cc:511] Master 127.13.109.62:38883 requested a full tablet report, sending...
I20260811 19:00:18.977909 13748 external_mini_cluster.cc:1428] Setting key a7e96ecd920858485dbb7bd55aa1e529
I20260811 19:00:18.979661 18741 ts_manager.cc:194] Registered new tserver with Master: 1ebaaf751ba34262ba390a70ddab9048 (127.13.109.1:40505)
I20260811 19:00:18.980083 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474792496492-13748-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474792496492-13748-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474792496492-13748-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.2:0
--local_ip_for_outbound_sockets=127.13.109.2
--webserver_interface=127.13.109.2
--webserver_port=0
--tserver_master_addrs=127.13.109.62:38883
--builtin_ntp_servers=127.13.109.20:34935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 19:00:19.101863 18947 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:00:19.102200 18947 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:00:19.102245 18947 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:00:19.106482 18947 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:00:19.106562 18947 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:00:19.106626 18947 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.2
I20260811 19:00:19.111863 18947 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:34935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474792496492-13748-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474792496492-13748-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.109.2
--webserver_port=0
--tserver_master_addrs=127.13.109.62:38883
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18947
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.2
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474792496492-13748-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:00:19.113073 18947 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:00:19.114631 18947 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:00:19.125273 18956 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:19.125229 18954 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:19.125229 18953 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:19.125855 18947 server_base.cc:1034] running on GCE node
I20260811 19:00:19.126319 18947 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:00:19.126977 18947 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:00:19.128140 18947 hybrid_clock.cc:648] HybridClock initialized: now 1786474819128106 us; error 45 us; skew 500 ppm
I20260811 19:00:19.130295 18947 webserver.cc:460] Webserver started at http://127.13.109.2:37561/ using document root <none> and password file <none>
I20260811 19:00:19.130926 18947 fs_manager.cc:359] Metadata directory not provided
I20260811 19:00:19.130995 18947 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:00:19.131229 18947 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:00:19.133435 18947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474792496492-13748-0/minicluster-data/ts-1/data/instance:
uuid: "11f601a524e948758d09351d7842de12"
format_stamp: "Formatted at 2026-08-11 19:00:19 on dist-test-slave-0kls"
server_key: "1b8dfb826983945004327f7b994d7a07"
server_key_iv: "c991bd4656b7e9c9a55735581c2ce6cf"
server_key_version: "encryptionkey@0"
I20260811 19:00:19.134052 18947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474792496492-13748-0/minicluster-data/ts-1/wal/instance:
uuid: "11f601a524e948758d09351d7842de12"
format_stamp: "Formatted at 2026-08-11 19:00:19 on dist-test-slave-0kls"
server_key: "1b8dfb826983945004327f7b994d7a07"
server_key_iv: "c991bd4656b7e9c9a55735581c2ce6cf"
server_key_version: "encryptionkey@0"
I20260811 19:00:19.138056 18947 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 19:00:19.140740 18962 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:19.142088 18947 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 19:00:19.142210 18947 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474792496492-13748-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474792496492-13748-0/minicluster-data/ts-1/wal
uuid: "11f601a524e948758d09351d7842de12"
format_stamp: "Formatted at 2026-08-11 19:00:19 on dist-test-slave-0kls"
server_key: "1b8dfb826983945004327f7b994d7a07"
server_key_iv: "c991bd4656b7e9c9a55735581c2ce6cf"
server_key_version: "encryptionkey@0"
I20260811 19:00:19.142328 18947 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474792496492-13748-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474792496492-13748-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474792496492-13748-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:00:19.147025 18947 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:00:19.147727 18947 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:00:19.147918 18947 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:00:19.148491 18947 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:00:19.149444 18947 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:00:19.149492 18947 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:19.149562 18947 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:00:19.149600 18947 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:19.159206 18947 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.2:44089
I20260811 19:00:19.159235 19085 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.2:44089 every 8 connection(s)
I20260811 19:00:19.160171 18947 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
I20260811 19:00:19.166980 18785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:19.162236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.2:44943 (local address 127.13.109.62:38883)
0811 19:00:19.162390 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:19.162393 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:19.163572 (+ 1179us) server_negotiation.cc:407] Connection header received
0811 19:00:19.164972 (+ 1400us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:19.164976 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:19.165026 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:19.165125 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:19.166429 (+ 1304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:19.166433 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:19.166436 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:00:19.166450 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:19.166475 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:00:19.166489 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:19.166598 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:19.166767 (+ 169us) server_negotiation.cc:299] Negotiation successful
0811 19:00:19.166823 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 19:00:19.167999 19089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:19.162691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:38883 (local address 127.13.109.2:44943)
0811 19:00:19.163408 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:19.163450 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:19.164750 (+ 1300us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:19.165314 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:19.165322 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:19.165425 (+ 103us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:19.166136 (+ 711us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 19:00:19.166138 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:19.166185 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 19:00:19.166265 (+ 80us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:00:19.166600 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:00:19.166612 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:00:19.166620 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 19:00:19.166825 (+ 205us) client_negotiation.cc:240] Negotiation successful
0811 19:00:19.167116 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":153,"threads_started":1}
I20260811 19:00:19.168242 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 18947
I20260811 19:00:19.168388 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474792496492-13748-0/minicluster-data/ts-1/wal/instance
I20260811 19:00:19.168741 13748 external_mini_cluster.cc:1428] Setting key 31a7d1a843a9be7a2e185551b367502d
I20260811 19:00:19.169622 19087 heartbeater.cc:347] Connected to a master server at 127.13.109.62:38883
I20260811 19:00:19.169927 19087 heartbeater.cc:464] Registering TS with master...
I20260811 19:00:19.170431 19087 heartbeater.cc:511] Master 127.13.109.62:38883 requested a full tablet report, sending...
I20260811 19:00:19.171145 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474792496492-13748-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474792496492-13748-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474792496492-13748-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.3:0
--local_ip_for_outbound_sockets=127.13.109.3
--webserver_interface=127.13.109.3
--webserver_port=0
--tserver_master_addrs=127.13.109.62:38883
--builtin_ntp_servers=127.13.109.20:34935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260811 19:00:19.171641 18741 ts_manager.cc:194] Registered new tserver with Master: 11f601a524e948758d09351d7842de12 (127.13.109.2:44089)
W20260811 19:00:19.291450 19090 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:00:19.291792 19090 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:00:19.291889 19090 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:00:19.295954 19090 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:00:19.296008 19090 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:00:19.296064 19090 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.3
I20260811 19:00:19.300819 19090 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:34935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474792496492-13748-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474792496492-13748-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.109.3
--webserver_port=0
--tserver_master_addrs=127.13.109.62:38883
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19090
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.3
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474792496492-13748-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:00:19.302040 19090 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:00:19.303414 19090 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:00:19.313359 19098 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:19.313408 19090 server_base.cc:1034] running on GCE node
W20260811 19:00:19.313378 19096 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:19.313354 19095 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:19.314044 19090 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:00:19.314747 19090 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:00:19.315918 19090 hybrid_clock.cc:648] HybridClock initialized: now 1786474819315894 us; error 35 us; skew 500 ppm
I20260811 19:00:19.317719 19090 webserver.cc:460] Webserver started at http://127.13.109.3:42187/ using document root <none> and password file <none>
I20260811 19:00:19.318331 19090 fs_manager.cc:359] Metadata directory not provided
I20260811 19:00:19.318395 19090 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:00:19.318662 19090 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:00:19.320745 19090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474792496492-13748-0/minicluster-data/ts-2/data/instance:
uuid: "bb797f5d34f4472dac7abbf7909cce35"
format_stamp: "Formatted at 2026-08-11 19:00:19 on dist-test-slave-0kls"
server_key: "a89832878185f001b334449417c97784"
server_key_iv: "dad4b9a990019caff2ac6e2e435ac41c"
server_key_version: "encryptionkey@0"
I20260811 19:00:19.321381 19090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474792496492-13748-0/minicluster-data/ts-2/wal/instance:
uuid: "bb797f5d34f4472dac7abbf7909cce35"
format_stamp: "Formatted at 2026-08-11 19:00:19 on dist-test-slave-0kls"
server_key: "a89832878185f001b334449417c97784"
server_key_iv: "dad4b9a990019caff2ac6e2e435ac41c"
server_key_version: "encryptionkey@0"
I20260811 19:00:19.325258 19090 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 19:00:19.327976 19104 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:19.329378 19090 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 19:00:19.329478 19090 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474792496492-13748-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474792496492-13748-0/minicluster-data/ts-2/wal
uuid: "bb797f5d34f4472dac7abbf7909cce35"
format_stamp: "Formatted at 2026-08-11 19:00:19 on dist-test-slave-0kls"
server_key: "a89832878185f001b334449417c97784"
server_key_iv: "dad4b9a990019caff2ac6e2e435ac41c"
server_key_version: "encryptionkey@0"
I20260811 19:00:19.329576 19090 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474792496492-13748-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474792496492-13748-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474792496492-13748-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:00:19.334203 19090 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:00:19.334789 19090 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:00:19.334934 19090 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:00:19.335429 19090 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:00:19.336393 19090 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:00:19.336442 19090 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:19.336480 19090 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:00:19.336509 19090 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:19.345649 19090 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.3:39627
I20260811 19:00:19.345762 19224 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.3:39627 every 8 connection(s)
I20260811 19:00:19.346755 19090 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
I20260811 19:00:19.349565 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 19090
I20260811 19:00:19.349675 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474792496492-13748-0/minicluster-data/ts-2/wal/instance
I20260811 19:00:19.350147 13748 external_mini_cluster.cc:1428] Setting key 82b218adabafda2b991e6ebe3de35dae
I20260811 19:00:19.353562 18785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:19.348730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.3:36487 (local address 127.13.109.62:38883)
0811 19:00:19.348909 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:19.348912 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:19.350037 (+ 1125us) server_negotiation.cc:407] Connection header received
0811 19:00:19.351750 (+ 1713us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:19.351753 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:19.351795 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:19.351876 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:19.353115 (+ 1239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:19.353118 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:19.353120 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:00:19.353133 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:19.353154 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:00:19.353161 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:19.353235 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:19.353366 (+ 131us) server_negotiation.cc:299] Negotiation successful
0811 19:00:19.353404 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 19:00:19.354589 19227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:19.349119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:38883 (local address 127.13.109.3:36487)
0811 19:00:19.349879 (+ 760us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:19.349931 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:19.351513 (+ 1582us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:19.352096 (+ 583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:19.352113 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:19.352233 (+ 120us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:19.352857 (+ 624us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 19:00:19.352859 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:19.352901 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 19:00:19.352973 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:00:19.353242 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:00:19.353248 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:00:19.353253 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 19:00:19.353391 (+ 138us) client_negotiation.cc:240] Negotiation successful
0811 19:00:19.353653 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":141,"threads_started":1}
I20260811 19:00:19.355712 19225 heartbeater.cc:347] Connected to a master server at 127.13.109.62:38883
I20260811 19:00:19.355953 19225 heartbeater.cc:464] Registering TS with master...
I20260811 19:00:19.356407 19225 heartbeater.cc:511] Master 127.13.109.62:38883 requested a full tablet report, sending...
I20260811 19:00:19.357470 18741 ts_manager.cc:194] Registered new tserver with Master: bb797f5d34f4472dac7abbf7909cce35 (127.13.109.3:39627)
I20260811 19:00:19.364784 13748 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 19:00:19.368713 18785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:19.366904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37566 (local address 127.13.109.62:38883)
0811 19:00:19.367073 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:19.367078 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:19.367250 (+ 172us) server_negotiation.cc:407] Connection header received
0811 19:00:19.367408 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:19.367411 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:19.367457 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:19.367544 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:19.368250 (+ 706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:19.368253 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:19.368256 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:00:19.368269 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:19.368291 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:00:19.368306 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:19.368397 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:19.368506 (+ 109us) server_negotiation.cc:299] Negotiation successful
0811 19:00:19.368549 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 19:00:19.371909 18741 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37566:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 19:00:19.375121 18741 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 19:00:19.387722 19237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:19.384104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.1:40505 (local address 127.0.0.1:49310)
0811 19:00:19.385097 (+ 993us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:19.385123 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:19.385213 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:19.386445 (+ 1232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:19.386447 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:19.386462 (+ 15us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:19.386946 (+ 484us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 19:00:19.386950 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:19.386963 (+ 13us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 19:00:19.386998 (+ 35us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:00:19.387323 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:00:19.387332 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:00:19.387338 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 19:00:19.387451 (+ 113us) client_negotiation.cc:240] Negotiation successful
0811 19:00:19.387545 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":926,"thread_start_us":109,"threads_started":1}
I20260811 19:00:19.388085 19238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:19.384158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44960 (local address 127.13.109.3:39627)
0811 19:00:19.385267 (+ 1109us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:19.385279 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:19.385311 (+ 32us) server_negotiation.cc:407] Connection header received
0811 19:00:19.385382 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:19.385388 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:19.385589 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:19.386203 (+ 614us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:19.387091 (+ 888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:19.387099 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:19.387114 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:00:19.387127 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:19.387172 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:00:19.387195 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:19.387324 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:19.387843 (+ 519us) server_negotiation.cc:299] Negotiation successful
0811 19:00:19.387928 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1007,"thread_start_us":170,"threads_started":1}
I20260811 19:00:19.387724 19236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:19.384026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.3:39627 (local address 127.0.0.1:44960)
0811 19:00:19.384853 (+ 827us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:19.384903 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:19.385000 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:19.385819 (+ 819us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:19.385825 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:19.385909 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:19.386489 (+ 580us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 19:00:19.386491 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:19.386510 (+ 19us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 19:00:19.386583 (+ 73us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:00:19.387387 (+ 804us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:00:19.387392 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:00:19.387396 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 19:00:19.387477 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 19:00:19.387559 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":634,"mutex_wait_us":24,"thread_start_us":94,"threads_started":1}
I20260811 19:00:19.388331 19239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:19.384516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49310 (local address 127.13.109.1:40505)
0811 19:00:19.384945 (+ 429us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:19.384959 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:19.385317 (+ 358us) server_negotiation.cc:407] Connection header received
0811 19:00:19.385382 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:19.385388 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:19.385588 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:19.385725 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:19.387123 (+ 1398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:19.387130 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:19.387141 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:00:19.387154 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:19.387200 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:00:19.387221 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:19.387338 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:19.388089 (+ 751us) server_negotiation.cc:299] Negotiation successful
0811 19:00:19.388183 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":119,"threads_started":1}
I20260811 19:00:19.388342 19241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:19.385184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52134 (local address 127.13.109.2:44089)
0811 19:00:19.385523 (+ 339us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:19.385538 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:19.385694 (+ 156us) server_negotiation.cc:407] Connection header received
0811 19:00:19.385756 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:19.385762 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:19.385958 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:19.386101 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:19.387575 (+ 1474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:19.387580 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:19.387590 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:00:19.387603 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:19.387654 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:00:19.387683 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:19.387785 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:19.388144 (+ 359us) server_negotiation.cc:299] Negotiation successful
0811 19:00:19.388203 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":243,"thread_start_us":129,"threads_started":1}
I20260811 19:00:19.388058 19240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:19.384769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.2:44089 (local address 127.0.0.1:52134)
0811 19:00:19.385571 (+ 802us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:19.385585 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:19.385682 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:19.386109 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:19.386114 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:19.386444 (+ 330us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:19.387398 (+ 954us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 19:00:19.387403 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:19.387416 (+ 13us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 19:00:19.387443 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:00:19.387835 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:00:19.387839 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:00:19.387845 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 19:00:19.387920 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 19:00:19.387951 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":743,"thread_start_us":84,"threads_started":1}
I20260811 19:00:19.391707 19158 tablet_service.cc:1467] Processing CreateTablet for tablet 1fe8e0b0076d4198af2978a5251b2a22 (DEFAULT_TABLE table=test-table [id=241ae3ec02894887b985e650c6956b03]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:00:19.391812 18875 tablet_service.cc:1467] Processing CreateTablet for tablet 1fe8e0b0076d4198af2978a5251b2a22 (DEFAULT_TABLE table=test-table [id=241ae3ec02894887b985e650c6956b03]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:00:19.392755 19158 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1fe8e0b0076d4198af2978a5251b2a22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:00:19.392757 18875 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1fe8e0b0076d4198af2978a5251b2a22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:00:19.399071 19242 tablet_bootstrap.cc:492] T 1fe8e0b0076d4198af2978a5251b2a22 P bb797f5d34f4472dac7abbf7909cce35: Bootstrap starting.
I20260811 19:00:19.399770 19015 tablet_service.cc:1467] Processing CreateTablet for tablet 1fe8e0b0076d4198af2978a5251b2a22 (DEFAULT_TABLE table=test-table [id=241ae3ec02894887b985e650c6956b03]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:00:19.400800 19243 tablet_bootstrap.cc:492] T 1fe8e0b0076d4198af2978a5251b2a22 P 1ebaaf751ba34262ba390a70ddab9048: Bootstrap starting.
I20260811 19:00:19.400931 19015 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1fe8e0b0076d4198af2978a5251b2a22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:00:19.402623 19242 tablet_bootstrap.cc:654] T 1fe8e0b0076d4198af2978a5251b2a22 P bb797f5d34f4472dac7abbf7909cce35: Neither blocks nor log segments found. Creating new log.
I20260811 19:00:19.402927 19243 tablet_bootstrap.cc:654] T 1fe8e0b0076d4198af2978a5251b2a22 P 1ebaaf751ba34262ba390a70ddab9048: Neither blocks nor log segments found. Creating new log.
I20260811 19:00:19.403575 19242 log.cc:824] T 1fe8e0b0076d4198af2978a5251b2a22 P bb797f5d34f4472dac7abbf7909cce35: Log is configured to *not* fsync() on all Append() calls
I20260811 19:00:19.403661 19243 log.cc:824] T 1fe8e0b0076d4198af2978a5251b2a22 P 1ebaaf751ba34262ba390a70ddab9048: Log is configured to *not* fsync() on all Append() calls
I20260811 19:00:19.405462 19243 tablet_bootstrap.cc:492] T 1fe8e0b0076d4198af2978a5251b2a22 P 1ebaaf751ba34262ba390a70ddab9048: No bootstrap required, opened a new log
I20260811 19:00:19.405730 19243 ts_tablet_manager.cc:1397] T 1fe8e0b0076d4198af2978a5251b2a22 P 1ebaaf751ba34262ba390a70ddab9048: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260811 19:00:19.405855 19242 tablet_bootstrap.cc:492] T 1fe8e0b0076d4198af2978a5251b2a22 P bb797f5d34f4472dac7abbf7909cce35: No bootstrap required, opened a new log
I20260811 19:00:19.406119 19242 ts_tablet_manager.cc:1397] T 1fe8e0b0076d4198af2978a5251b2a22 P bb797f5d34f4472dac7abbf7909cce35: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 19:00:19.408002 19246 tablet_bootstrap.cc:492] T 1fe8e0b0076d4198af2978a5251b2a22 P 11f601a524e948758d09351d7842de12: Bootstrap starting.
I20260811 19:00:19.409931 19243 raft_consensus.cc:348] T 1fe8e0b0076d4198af2978a5251b2a22 P 1ebaaf751ba34262ba390a70ddab9048 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb797f5d34f4472dac7abbf7909cce35" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 39627 } } peers { permanent_uuid: "1ebaaf751ba34262ba390a70ddab9048" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 40505 } } peers { permanent_uuid: "11f601a524e948758d09351d7842de12" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 44089 } }
I20260811 19:00:19.410228 19243 raft_consensus.cc:374] T 1fe8e0b0076d4198af2978a5251b2a22 P 1ebaaf751ba34262ba390a70ddab9048 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:00:19.410300 19243 raft_consensus.cc:729] T 1fe8e0b0076d4198af2978a5251b2a22 P 1ebaaf751ba34262ba390a70ddab9048 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ebaaf751ba34262ba390a70ddab9048, State: Initialized, Role: FOLLOWER
I20260811 19:00:19.410272 19242 raft_consensus.cc:348] T 1fe8e0b0076d4198af2978a5251b2a22 P bb797f5d34f4472dac7abbf7909cce35 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb797f5d34f4472dac7abbf7909cce35" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 39627 } } peers { permanent_uuid: "1ebaaf751ba34262ba390a70ddab9048" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 40505 } } peers { permanent_uuid: "11f601a524e948758d09351d7842de12" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 44089 } }
I20260811 19:00:19.410493 19246 tablet_bootstrap.cc:654] T 1fe8e0b0076d4198af2978a5251b2a22 P 11f601a524e948758d09351d7842de12: Neither blocks nor log segments found. Creating new log.
I20260811 19:00:19.410553 19242 raft_consensus.cc:374] T 1fe8e0b0076d4198af2978a5251b2a22 P bb797f5d34f4472dac7abbf7909cce35 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:00:19.410624 19242 raft_consensus.cc:729] T 1fe8e0b0076d4198af2978a5251b2a22 P bb797f5d34f4472dac7abbf7909cce35 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb797f5d34f4472dac7abbf7909cce35, State: Initialized, Role: FOLLOWER
I20260811 19:00:19.410955 19243 consensus_queue.cc:260] T 1fe8e0b0076d4198af2978a5251b2a22 P 1ebaaf751ba34262ba390a70ddab9048 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb797f5d34f4472dac7abbf7909cce35" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 39627 } } peers { permanent_uuid: "1ebaaf751ba34262ba390a70ddab9048" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 40505 } } peers { permanent_uuid: "11f601a524e948758d09351d7842de12" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 44089 } }
I20260811 19:00:19.411067 19242 consensus_queue.cc:260] T 1fe8e0b0076d4198af2978a5251b2a22 P bb797f5d34f4472dac7abbf7909cce35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb797f5d34f4472dac7abbf7909cce35" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 39627 } } peers { permanent_uuid: "1ebaaf751ba34262ba390a70ddab9048" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 40505 } } peers { permanent_uuid: "11f601a524e948758d09351d7842de12" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 44089 } }
I20260811 19:00:19.411441 19246 log.cc:824] T 1fe8e0b0076d4198af2978a5251b2a22 P 11f601a524e948758d09351d7842de12: Log is configured to *not* fsync() on all Append() calls
I20260811 19:00:19.411782 18944 heartbeater.cc:503] Master 127.13.109.62:38883 was elected leader, sending a full tablet report...
I20260811 19:00:19.412082 19242 ts_tablet_manager.cc:1428] T 1fe8e0b0076d4198af2978a5251b2a22 P bb797f5d34f4472dac7abbf7909cce35: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260811 19:00:19.412109 19243 ts_tablet_manager.cc:1428] T 1fe8e0b0076d4198af2978a5251b2a22 P 1ebaaf751ba34262ba390a70ddab9048: Time spent starting tablet: real 0.006s user 0.004s sys 0.002s
I20260811 19:00:19.412406 19225 heartbeater.cc:503] Master 127.13.109.62:38883 was elected leader, sending a full tablet report...
I20260811 19:00:19.413712 19246 tablet_bootstrap.cc:492] T 1fe8e0b0076d4198af2978a5251b2a22 P 11f601a524e948758d09351d7842de12: No bootstrap required, opened a new log
I20260811 19:00:19.413959 19246 ts_tablet_manager.cc:1397] T 1fe8e0b0076d4198af2978a5251b2a22 P 11f601a524e948758d09351d7842de12: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20260811 19:00:19.418104 19246 raft_consensus.cc:348] T 1fe8e0b0076d4198af2978a5251b2a22 P 11f601a524e948758d09351d7842de12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb797f5d34f4472dac7abbf7909cce35" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 39627 } } peers { permanent_uuid: "1ebaaf751ba34262ba390a70ddab9048" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 40505 } } peers { permanent_uuid: "11f601a524e948758d09351d7842de12" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 44089 } }
I20260811 19:00:19.418378 19246 raft_consensus.cc:374] T 1fe8e0b0076d4198af2978a5251b2a22 P 11f601a524e948758d09351d7842de12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:00:19.418462 19246 raft_consensus.cc:729] T 1fe8e0b0076d4198af2978a5251b2a22 P 11f601a524e948758d09351d7842de12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11f601a524e948758d09351d7842de12, State: Initialized, Role: FOLLOWER
I20260811 19:00:19.418907 19246 consensus_queue.cc:260] T 1fe8e0b0076d4198af2978a5251b2a22 P 11f601a524e948758d09351d7842de12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb797f5d34f4472dac7abbf7909cce35" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 39627 } } peers { permanent_uuid: "1ebaaf751ba34262ba390a70ddab9048" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 40505 } } peers { permanent_uuid: "11f601a524e948758d09351d7842de12" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 44089 } }
I20260811 19:00:19.419590 19087 heartbeater.cc:503] Master 127.13.109.62:38883 was elected leader, sending a full tablet report...
I20260811 19:00:19.419971 19246 ts_tablet_manager.cc:1428] T 1fe8e0b0076d4198af2978a5251b2a22 P 11f601a524e948758d09351d7842de12: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260811 19:00:19.461335 19247 raft_consensus.cc:482] T 1fe8e0b0076d4198af2978a5251b2a22 P 1ebaaf751ba34262ba390a70ddab9048 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:00:19.461623 19247 raft_consensus.cc:504] T 1fe8e0b0076d4198af2978a5251b2a22 P 1ebaaf751ba34262ba390a70ddab9048 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb797f5d34f4472dac7abbf7909cce35" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 39627 } } peers { permanent_uuid: "1ebaaf751ba34262ba390a70ddab9048" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 40505 } } peers { permanent_uuid: "11f601a524e948758d09351d7842de12" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 44089 } }
I20260811 19:00:19.462934 19247 leader_election.cc:288] T 1fe8e0b0076d4198af2978a5251b2a22 P 1ebaaf751ba34262ba390a70ddab9048 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb797f5d34f4472dac7abbf7909cce35 (127.13.109.3:39627), 11f601a524e948758d09351d7842de12 (127.13.109.2:44089)
I20260811 19:00:19.465116 18946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:19.463161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.2:44089 (local address 127.13.109.1:53503)
0811 19:00:19.463289 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:19.463306 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:19.463392 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:19.463727 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:19.463730 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:19.463746 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:19.464431 (+ 685us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 19:00:19.464433 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:19.464448 (+ 15us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 19:00:19.464480 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:00:19.464840 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:00:19.464845 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:00:19.464849 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 19:00:19.464944 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 19:00:19.464987 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260811 19:00:19.465168 19241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:19.463317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.1:53503 (local address 127.13.109.2:44089)
0811 19:00:19.463460 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:19.463467 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:19.463483 (+ 16us) server_negotiation.cc:407] Connection header received
0811 19:00:19.463530 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:19.463533 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:19.463578 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:19.463675 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:19.464655 (+ 980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:19.464660 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:19.464664 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:00:19.464681 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:19.464707 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:00:19.464719 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:19.464825 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:19.464960 (+ 135us) server_negotiation.cc:299] Negotiation successful
0811 19:00:19.465030 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 19:00:19.465574 19238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:19.463298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.1:60167 (local address 127.13.109.3:39627)
0811 19:00:19.463444 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:19.463448 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:19.463999 (+ 551us) server_negotiation.cc:407] Connection header received
0811 19:00:19.464061 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:19.464067 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:19.464112 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:19.464208 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:19.465054 (+ 846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:19.465058 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:19.465060 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:00:19.465072 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:19.465116 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:00:19.465126 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:19.465223 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:19.465400 (+ 177us) server_negotiation.cc:299] Negotiation successful
0811 19:00:19.465446 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 19:00:19.465973 19252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:19.463167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.3:39627 (local address 127.13.109.1:60167)
0811 19:00:19.463863 (+ 696us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:19.463879 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:19.463992 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:19.464293 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:19.464299 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:19.464324 (+ 25us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:19.464872 (+ 548us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 19:00:19.464876 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:19.464890 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 19:00:19.464920 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:00:19.465272 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:00:19.465275 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:00:19.465278 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 19:00:19.465766 (+ 488us) client_negotiation.cc:240] Negotiation successful
0811 19:00:19.465836 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":609,"mutex_wait_us":300,"thread_start_us":116,"threads_started":1}
I20260811 19:00:19.465919 19037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fe8e0b0076d4198af2978a5251b2a22" candidate_uuid: "1ebaaf751ba34262ba390a70ddab9048" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f601a524e948758d09351d7842de12" is_pre_election: true
I20260811 19:00:19.466252 19037 raft_consensus.cc:2454] T 1fe8e0b0076d4198af2978a5251b2a22 P 11f601a524e948758d09351d7842de12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ebaaf751ba34262ba390a70ddab9048 in term 0.
I20260811 19:00:19.466601 19179 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fe8e0b0076d4198af2978a5251b2a22" candidate_uuid: "1ebaaf751ba34262ba390a70ddab9048" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb797f5d34f4472dac7abbf7909cce35" is_pre_election: true
I20260811 19:00:19.466771 18824 leader_election.cc:302] T 1fe8e0b0076d4198af2978a5251b2a22 P 1ebaaf751ba34262ba390a70ddab9048 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f601a524e948758d09351d7842de12, 1ebaaf751ba34262ba390a70ddab9048; no voters:
I20260811 19:00:19.466893 19179 raft_consensus.cc:2454] T 1fe8e0b0076d4198af2978a5251b2a22 P bb797f5d34f4472dac7abbf7909cce35 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ebaaf751ba34262ba390a70ddab9048 in term 0.
I20260811 19:00:19.467103 19247 raft_consensus.cc:2781] T 1fe8e0b0076d4198af2978a5251b2a22 P 1ebaaf751ba34262ba390a70ddab9048 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 19:00:19.467190 19247 raft_consensus.cc:482] T 1fe8e0b0076d4198af2978a5251b2a22 P 1ebaaf751ba34262ba390a70ddab9048 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 19:00:19.467242 19247 raft_consensus.cc:3037] T 1fe8e0b0076d4198af2978a5251b2a22 P 1ebaaf751ba34262ba390a70ddab9048 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:00:19.468300 19247 raft_consensus.cc:504] T 1fe8e0b0076d4198af2978a5251b2a22 P 1ebaaf751ba34262ba390a70ddab9048 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb797f5d34f4472dac7abbf7909cce35" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 39627 } } peers { permanent_uuid: "1ebaaf751ba34262ba390a70ddab9048" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 40505 } } peers { permanent_uuid: "11f601a524e948758d09351d7842de12" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 44089 } }
I20260811 19:00:19.468653 19247 leader_election.cc:288] T 1fe8e0b0076d4198af2978a5251b2a22 P 1ebaaf751ba34262ba390a70ddab9048 [CANDIDATE]: Term 1 election: Requested vote from peers bb797f5d34f4472dac7abbf7909cce35 (127.13.109.3:39627), 11f601a524e948758d09351d7842de12 (127.13.109.2:44089)
I20260811 19:00:19.469010 19037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fe8e0b0076d4198af2978a5251b2a22" candidate_uuid: "1ebaaf751ba34262ba390a70ddab9048" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f601a524e948758d09351d7842de12"
I20260811 19:00:19.469002 19179 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fe8e0b0076d4198af2978a5251b2a22" candidate_uuid: "1ebaaf751ba34262ba390a70ddab9048" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb797f5d34f4472dac7abbf7909cce35"
W20260811 19:00:19.469149 18945 tablet.cc:2369] T 1fe8e0b0076d4198af2978a5251b2a22 P 1ebaaf751ba34262ba390a70ddab9048: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 19:00:19.469152 19179 raft_consensus.cc:3037] T 1fe8e0b0076d4198af2978a5251b2a22 P bb797f5d34f4472dac7abbf7909cce35 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:00:19.469167 19037 raft_consensus.cc:3037] T 1fe8e0b0076d4198af2978a5251b2a22 P 11f601a524e948758d09351d7842de12 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:00:19.470571 19179 raft_consensus.cc:2454] T 1fe8e0b0076d4198af2978a5251b2a22 P bb797f5d34f4472dac7abbf7909cce35 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ebaaf751ba34262ba390a70ddab9048 in term 1.
I20260811 19:00:19.470571 19037 raft_consensus.cc:2454] T 1fe8e0b0076d4198af2978a5251b2a22 P 11f601a524e948758d09351d7842de12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ebaaf751ba34262ba390a70ddab9048 in term 1.
I20260811 19:00:19.470933 18825 leader_election.cc:302] T 1fe8e0b0076d4198af2978a5251b2a22 P 1ebaaf751ba34262ba390a70ddab9048 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 11f601a524e948758d09351d7842de12, 1ebaaf751ba34262ba390a70ddab9048, bb797f5d34f4472dac7abbf7909cce35; no voters:
I20260811 19:00:19.471144 19247 raft_consensus.cc:2781] T 1fe8e0b0076d4198af2978a5251b2a22 P 1ebaaf751ba34262ba390a70ddab9048 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:00:19.471400 19247 raft_consensus.cc:686] T 1fe8e0b0076d4198af2978a5251b2a22 P 1ebaaf751ba34262ba390a70ddab9048 [term 1 LEADER]: Becoming Leader. State: Replica: 1ebaaf751ba34262ba390a70ddab9048, State: Running, Role: LEADER
I20260811 19:00:19.471764 19247 consensus_queue.cc:237] T 1fe8e0b0076d4198af2978a5251b2a22 P 1ebaaf751ba34262ba390a70ddab9048 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb797f5d34f4472dac7abbf7909cce35" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 39627 } } peers { permanent_uuid: "1ebaaf751ba34262ba390a70ddab9048" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 40505 } } peers { permanent_uuid: "11f601a524e948758d09351d7842de12" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 44089 } }
I20260811 19:00:19.475144 18742 catalog_manager.cc:5495] T 1fe8e0b0076d4198af2978a5251b2a22 P 1ebaaf751ba34262ba390a70ddab9048 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1ebaaf751ba34262ba390a70ddab9048 (127.13.109.1). New cstate: current_term: 1 leader_uuid: "1ebaaf751ba34262ba390a70ddab9048" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb797f5d34f4472dac7abbf7909cce35" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 39627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ebaaf751ba34262ba390a70ddab9048" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 40505 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f601a524e948758d09351d7842de12" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 44089 } health_report { overall_health: UNKNOWN } } }
I20260811 19:00:19.488044 19239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:19.486222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49324 (local address 127.13.109.1:40505)
0811 19:00:19.486402 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:19.486407 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:19.486424 (+ 17us) server_negotiation.cc:407] Connection header received
0811 19:00:19.486646 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:19.486648 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:19.486717 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:19.486823 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:19.487538 (+ 715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:19.487542 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:19.487544 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:00:19.487557 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:19.487581 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:00:19.487595 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:19.487696 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:19.487798 (+ 102us) server_negotiation.cc:299] Negotiation successful
0811 19:00:19.487842 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 19:00:19.494854 19179 raft_consensus.cc:1261] T 1fe8e0b0076d4198af2978a5251b2a22 P bb797f5d34f4472dac7abbf7909cce35 [term 1 FOLLOWER]: Refusing update from remote peer 1ebaaf751ba34262ba390a70ddab9048: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:00:19.494854 19037 raft_consensus.cc:1261] T 1fe8e0b0076d4198af2978a5251b2a22 P 11f601a524e948758d09351d7842de12 [term 1 FOLLOWER]: Refusing update from remote peer 1ebaaf751ba34262ba390a70ddab9048: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:00:19.495819 19253 consensus_queue.cc:1035] T 1fe8e0b0076d4198af2978a5251b2a22 P 1ebaaf751ba34262ba390a70ddab9048 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb797f5d34f4472dac7abbf7909cce35" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:00:19.496035 19247 consensus_queue.cc:1035] T 1fe8e0b0076d4198af2978a5251b2a22 P 1ebaaf751ba34262ba390a70ddab9048 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f601a524e948758d09351d7842de12" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 44089 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:00:19.507100 19258 mvcc.cc:205] Tried to move back new op lower bound from 7317400860642263040 to 7317400860559093760. Current Snapshot: MvccSnapshot[applied={T|T < 7317400860642263040}]
I20260811 19:00:19.507804 19256 mvcc.cc:205] Tried to move back new op lower bound from 7317400860642263040 to 7317400860559093760. Current Snapshot: MvccSnapshot[applied={T|T < 7317400860642263040 or (T in {7317400860642263040})}]
I20260811 19:00:19.513331 18741 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37566:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:00:19.513502 18741 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37566:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:00:19.516220 18741 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 13c92f3a1083414a9cc49f4604b90270: Sending DeleteTablet for 3 replicas of tablet 1fe8e0b0076d4198af2978a5251b2a22
I20260811 19:00:19.517182 19158 tablet_service.cc:1514] Processing DeleteTablet for tablet 1fe8e0b0076d4198af2978a5251b2a22 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:00:19 UTC) from {username='slave'} at 127.0.0.1:44960
I20260811 19:00:19.517249 18875 tablet_service.cc:1514] Processing DeleteTablet for tablet 1fe8e0b0076d4198af2978a5251b2a22 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:00:19 UTC) from {username='slave'} at 127.0.0.1:49310
I20260811 19:00:19.517750 19268 tablet_replica.cc:330] T 1fe8e0b0076d4198af2978a5251b2a22 P 1ebaaf751ba34262ba390a70ddab9048: stopping tablet replica
I20260811 19:00:19.517782 19015 tablet_service.cc:1514] Processing DeleteTablet for tablet 1fe8e0b0076d4198af2978a5251b2a22 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:00:19 UTC) from {username='slave'} at 127.0.0.1:52134
I20260811 19:00:19.517750 19267 tablet_replica.cc:330] T 1fe8e0b0076d4198af2978a5251b2a22 P bb797f5d34f4472dac7abbf7909cce35: stopping tablet replica
I20260811 19:00:19.518133 19268 raft_consensus.cc:2229] T 1fe8e0b0076d4198af2978a5251b2a22 P 1ebaaf751ba34262ba390a70ddab9048 [term 1 LEADER]: Raft consensus shutting down.
I20260811 19:00:19.518334 19267 raft_consensus.cc:2229] T 1fe8e0b0076d4198af2978a5251b2a22 P bb797f5d34f4472dac7abbf7909cce35 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:00:19.518545 19268 raft_consensus.cc:2258] T 1fe8e0b0076d4198af2978a5251b2a22 P 1ebaaf751ba34262ba390a70ddab9048 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:00:19.518816 19267 raft_consensus.cc:2258] T 1fe8e0b0076d4198af2978a5251b2a22 P bb797f5d34f4472dac7abbf7909cce35 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:00:19.519130 19269 tablet_replica.cc:330] T 1fe8e0b0076d4198af2978a5251b2a22 P 11f601a524e948758d09351d7842de12: stopping tablet replica
I20260811 19:00:19.519167 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 18807
I20260811 19:00:19.522132 19269 raft_consensus.cc:2229] T 1fe8e0b0076d4198af2978a5251b2a22 P 11f601a524e948758d09351d7842de12 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:00:19.522459 19269 raft_consensus.cc:2258] T 1fe8e0b0076d4198af2978a5251b2a22 P 11f601a524e948758d09351d7842de12 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:00:19.524127 19269 ts_tablet_manager.cc:1905] T 1fe8e0b0076d4198af2978a5251b2a22 P 11f601a524e948758d09351d7842de12: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:00:19.527416 19269 ts_tablet_manager.cc:1918] T 1fe8e0b0076d4198af2978a5251b2a22 P 11f601a524e948758d09351d7842de12: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:00:19.527557 19269 log.cc:1196] T 1fe8e0b0076d4198af2978a5251b2a22 P 11f601a524e948758d09351d7842de12: Deleting WAL directory at /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474792496492-13748-0/minicluster-data/ts-1/wal/wals/1fe8e0b0076d4198af2978a5251b2a22
I20260811 19:00:19.527905 19269 ts_tablet_manager.cc:1939] T 1fe8e0b0076d4198af2978a5251b2a22 P 11f601a524e948758d09351d7842de12: Deleting consensus metadata
I20260811 19:00:19.528903 19267 ts_tablet_manager.cc:1905] T 1fe8e0b0076d4198af2978a5251b2a22 P bb797f5d34f4472dac7abbf7909cce35: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:00:19.529839 18727 catalog_manager.cc:4841] TS 11f601a524e948758d09351d7842de12 (127.13.109.2:44089): tablet 1fe8e0b0076d4198af2978a5251b2a22 (table test-table [id=241ae3ec02894887b985e650c6956b03]) successfully deleted
I20260811 19:00:19.531690 19267 ts_tablet_manager.cc:1918] T 1fe8e0b0076d4198af2978a5251b2a22 P bb797f5d34f4472dac7abbf7909cce35: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:00:19.531810 19267 log.cc:1196] T 1fe8e0b0076d4198af2978a5251b2a22 P bb797f5d34f4472dac7abbf7909cce35: Deleting WAL directory at /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474792496492-13748-0/minicluster-data/ts-2/wal/wals/1fe8e0b0076d4198af2978a5251b2a22
I20260811 19:00:19.532128 19267 ts_tablet_manager.cc:1939] T 1fe8e0b0076d4198af2978a5251b2a22 P bb797f5d34f4472dac7abbf7909cce35: Deleting consensus metadata
I20260811 19:00:19.533324 18728 catalog_manager.cc:4841] TS bb797f5d34f4472dac7abbf7909cce35 (127.13.109.3:39627): tablet 1fe8e0b0076d4198af2978a5251b2a22 (table test-table [id=241ae3ec02894887b985e650c6956b03]) successfully deleted
W20260811 19:00:19.536375 18727 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.109.1:40505 (error 108)
I20260811 19:00:19.536579 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 18947
I20260811 19:00:19.538322 19237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:19.537884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.1:40505 (local address 127.0.0.1:49336)
0811 19:00:19.538126 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:19.538209 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.1:40505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128}
W20260811 19:00:19.538519 18727 catalog_manager.cc:4584] TS 1ebaaf751ba34262ba390a70ddab9048 (127.13.109.1:40505): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1fe8e0b0076d4198af2978a5251b2a22: Network error: Client connection negotiation failed: client connection to 127.13.109.1:40505: connect: Connection refused (error 111)
I20260811 19:00:19.544564 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 19090
I20260811 19:00:19.553143 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 18709
2026-08-11T19:00:19Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1202 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260811 19:00:19.578783 13748 mini_oidc.cc:152] Starting JWKS server
I20260811 19:00:19.580678 13748 webserver.cc:460] Webserver started at https://127.0.0.1:38317/ using document root <none> and password file <none>
I20260811 19:00:19.580842 13748 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 19:00:19.581110 13748 webserver.cc:460] Webserver started at http://127.0.0.1:42953/ using document root <none> and password file <none>
2026-08-11T19:00:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:00:19Z Disabled control of system clock
I20260811 19:00:19.592655 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474792496492-13748-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474792496492-13748-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474792496492-13748-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.62:44541
--webserver_interface=127.13.109.62
--webserver_port=0
--builtin_ntp_servers=127.13.109.20:35981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.109.62:44541
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38317/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474792496492-13748-0/testchainca.cert with env {}
W20260811 19:00:19.715565 19278 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:00:19.715857 19278 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:00:19.715899 19278 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:00:19.720023 19278 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:00:19.720090 19278 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:00:19.720112 19278 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:00:19.720130 19278 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:00:19.720146 19278 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 19:00:19.720163 19278 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 19:00:19.720177 19278 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:38317/jwks.json
I20260811 19:00:19.725775 19278 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:35981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474792496492-13748-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474792496492-13748-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.109.62:44541
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.62:44541
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38317/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.109.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474792496492-13748-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19278
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474792496492-13748-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:00:19.726913 19278 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:00:19.728035 19278 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:00:19.737645 19284 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:19.737674 19283 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:19.737650 19287 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:19.738173 19278 server_base.cc:1034] running on GCE node
I20260811 19:00:19.738909 19278 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:00:19.740157 19278 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:00:19.741381 19278 hybrid_clock.cc:648] HybridClock initialized: now 1786474819741363 us; error 34 us; skew 500 ppm
I20260811 19:00:19.743451 19278 webserver.cc:460] Webserver started at http://127.13.109.62:43889/ using document root <none> and password file <none>
I20260811 19:00:19.744082 19278 fs_manager.cc:359] Metadata directory not provided
I20260811 19:00:19.744329 19278 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:00:19.744604 19278 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:00:19.746810 19278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474792496492-13748-0/minicluster-data/master-0/data/instance:
uuid: "a913c5f58f554ae9ac829e0a3864bd30"
format_stamp: "Formatted at 2026-08-11 19:00:19 on dist-test-slave-0kls"
server_key: "d30e0b694cdb872ab1d0aa4783fe20e9"
server_key_iv: "6e2f8ecbe4a687111870c16b2671af09"
server_key_version: "encryptionkey@0"
I20260811 19:00:19.747396 19278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474792496492-13748-0/minicluster-data/master-0/wal/instance:
uuid: "a913c5f58f554ae9ac829e0a3864bd30"
format_stamp: "Formatted at 2026-08-11 19:00:19 on dist-test-slave-0kls"
server_key: "d30e0b694cdb872ab1d0aa4783fe20e9"
server_key_iv: "6e2f8ecbe4a687111870c16b2671af09"
server_key_version: "encryptionkey@0"
I20260811 19:00:19.751407 19278 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 19:00:19.754137 19294 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:19.755450 19278 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 19:00:19.755621 19278 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474792496492-13748-0/minicluster-data/master-0/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474792496492-13748-0/minicluster-data/master-0/wal
uuid: "a913c5f58f554ae9ac829e0a3864bd30"
format_stamp: "Formatted at 2026-08-11 19:00:19 on dist-test-slave-0kls"
server_key: "d30e0b694cdb872ab1d0aa4783fe20e9"
server_key_iv: "6e2f8ecbe4a687111870c16b2671af09"
server_key_version: "encryptionkey@0"
I20260811 19:00:19.755753 19278 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474792496492-13748-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474792496492-13748-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474792496492-13748-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:00:19.773675 19278 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:00:19.774557 19278 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:00:19.774796 19278 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:00:19.782773 19278 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.62:44541
I20260811 19:00:19.782783 19348 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.62:44541 every 8 connection(s)
I20260811 19:00:19.783838 19278 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
I20260811 19:00:19.787516 19349 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:00:19.789951 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 19278
I20260811 19:00:19.790096 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474792496492-13748-0/minicluster-data/master-0/wal/instance
I20260811 19:00:19.790402 13748 external_mini_cluster.cc:1428] Setting key f924214366f1ad009bfa806da9d40ac3
I20260811 19:00:19.794205 19349 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a913c5f58f554ae9ac829e0a3864bd30: Bootstrap starting.
I20260811 19:00:19.796782 19349 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a913c5f58f554ae9ac829e0a3864bd30: Neither blocks nor log segments found. Creating new log.
I20260811 19:00:19.797664 19349 log.cc:824] T 00000000000000000000000000000000 P a913c5f58f554ae9ac829e0a3864bd30: Log is configured to *not* fsync() on all Append() calls
I20260811 19:00:19.800370 19349 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a913c5f58f554ae9ac829e0a3864bd30: No bootstrap required, opened a new log
I20260811 19:00:19.800711 19352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:19.791887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43720 (local address 127.13.109.62:44541)
0811 19:00:19.792605 (+ 718us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:19.792621 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:19.792659 (+ 38us) server_negotiation.cc:407] Connection header received
0811 19:00:19.793333 (+ 674us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:19.793353 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:19.793866 (+ 513us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:19.794247 (+ 381us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:19.794731 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:19.795725 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:19.796694 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:19.797278 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:19.797469 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:19.798441 (+ 972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:19.798446 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:19.798461 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:00:19.798975 (+ 514us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:19.799052 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:00:19.799081 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:19.799218 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:19.799371 (+ 153us) server_negotiation.cc:299] Negotiation successful
0811 19:00:19.799664 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":148,"threads_started":1}
I20260811 19:00:19.803994 13748 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 19:00:19.805032 19349 raft_consensus.cc:348] T 00000000000000000000000000000000 P a913c5f58f554ae9ac829e0a3864bd30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a913c5f58f554ae9ac829e0a3864bd30" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 44541 } }
I20260811 19:00:19.805313 19349 raft_consensus.cc:374] T 00000000000000000000000000000000 P a913c5f58f554ae9ac829e0a3864bd30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:00:19.805388 19349 raft_consensus.cc:729] T 00000000000000000000000000000000 P a913c5f58f554ae9ac829e0a3864bd30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a913c5f58f554ae9ac829e0a3864bd30, State: Initialized, Role: FOLLOWER
I20260811 19:00:19.805917 19349 consensus_queue.cc:260] T 00000000000000000000000000000000 P a913c5f58f554ae9ac829e0a3864bd30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a913c5f58f554ae9ac829e0a3864bd30" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 44541 } }
I20260811 19:00:19.806078 19349 raft_consensus.cc:388] T 00000000000000000000000000000000 P a913c5f58f554ae9ac829e0a3864bd30 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:00:19.806124 19349 raft_consensus.cc:482] T 00000000000000000000000000000000 P a913c5f58f554ae9ac829e0a3864bd30 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:00:19.806205 19349 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a913c5f58f554ae9ac829e0a3864bd30 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:00:19.807173 19349 raft_consensus.cc:504] T 00000000000000000000000000000000 P a913c5f58f554ae9ac829e0a3864bd30 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a913c5f58f554ae9ac829e0a3864bd30" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 44541 } }
I20260811 19:00:19.807502 19349 leader_election.cc:302] T 00000000000000000000000000000000 P a913c5f58f554ae9ac829e0a3864bd30 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a913c5f58f554ae9ac829e0a3864bd30; no voters:
I20260811 19:00:19.807799 19349 leader_election.cc:288] T 00000000000000000000000000000000 P a913c5f58f554ae9ac829e0a3864bd30 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:00:19.807941 19355 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a913c5f58f554ae9ac829e0a3864bd30 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:00:19.808209 19355 raft_consensus.cc:686] T 00000000000000000000000000000000 P a913c5f58f554ae9ac829e0a3864bd30 [term 1 LEADER]: Becoming Leader. State: Replica: a913c5f58f554ae9ac829e0a3864bd30, State: Running, Role: LEADER
I20260811 19:00:19.808563 19355 consensus_queue.cc:237] T 00000000000000000000000000000000 P a913c5f58f554ae9ac829e0a3864bd30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a913c5f58f554ae9ac829e0a3864bd30" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 44541 } }
I20260811 19:00:19.809266 19349 sys_catalog.cc:564] T 00000000000000000000000000000000 P a913c5f58f554ae9ac829e0a3864bd30 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:00:19.810343 19356 sys_catalog.cc:455] T 00000000000000000000000000000000 P a913c5f58f554ae9ac829e0a3864bd30 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a913c5f58f554ae9ac829e0a3864bd30" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a913c5f58f554ae9ac829e0a3864bd30" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 44541 } } }
I20260811 19:00:19.810510 19356 sys_catalog.cc:458] T 00000000000000000000000000000000 P a913c5f58f554ae9ac829e0a3864bd30 [sys.catalog]: This master's current role is: LEADER
I20260811 19:00:19.810827 19357 sys_catalog.cc:455] T 00000000000000000000000000000000 P a913c5f58f554ae9ac829e0a3864bd30 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a913c5f58f554ae9ac829e0a3864bd30. Latest consensus state: current_term: 1 leader_uuid: "a913c5f58f554ae9ac829e0a3864bd30" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a913c5f58f554ae9ac829e0a3864bd30" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 44541 } } }
I20260811 19:00:19.810942 19357 sys_catalog.cc:458] T 00000000000000000000000000000000 P a913c5f58f554ae9ac829e0a3864bd30 [sys.catalog]: This master's current role is: LEADER
I20260811 19:00:19.811053 19364 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:00:19.815044 19364 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:00:19.821692 19364 catalog_manager.cc:1347] Generated new cluster ID: 3cd687380d1348c0bc9e1a8d5007429a
I20260811 19:00:19.821810 19364 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:00:19.835260 19364 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:00:19.836771 19364 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:00:19.844578 19364 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a913c5f58f554ae9ac829e0a3864bd30: Generated new TSK 0
I20260811 19:00:19.845443 19364 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260811 19:00:19.886332 19380 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260811 19:00:19.886526 19380 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 19:00:19.887207 19352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:19.867508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43732 (local address 127.13.109.62:44541)
0811 19:00:19.867650 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:19.867655 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:19.867865 (+ 210us) server_negotiation.cc:407] Connection header received
0811 19:00:19.868140 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:19.868143 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:19.868193 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:19.868303 (+ 110us) server_negotiation.cc:226] Negotiated authn=JWT
0811 19:00:19.868575 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:19.869121 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:19.870109 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:19.870351 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:19.870467 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:19.870841 (+ 374us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 19:00:19.886682 (+ 15841us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38317/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0811 19:00:19.887027 (+ 345us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:43732: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38317/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":53}
W20260811 19:00:19.887315 19352 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:43732: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38317/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260811 19:00:19.888574 19379 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:00:19.867399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:44541 (local address 127.0.0.1:43732)
0811 19:00:19.867778 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:19.867790 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:19.867940 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:19.868321 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:19.868324 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:19.868332 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0811 19:00:19.868463 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:19.868468 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:19.869235 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:19.869239 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:19.869974 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:19.869982 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:19.870478 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:19.870482 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:19.870584 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:19.870637 (+ 53us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 19:00:19.886956 (+ 16319us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38317/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0811 19:00:19.887248 (+ 292us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.13.109.62:44541: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38317/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":125,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.13.109.62:44541: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38317/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260811 19:00:19.889506 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 19278
2026-08-11T19:00:19Z chronyd exiting
I20260811 19:00:19.929955 13748 test_util.cc:175] -----------------------------------------------
I20260811 19:00:19.930048 13748 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474792496492-13748-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (353 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260811 19:00:19.931133 13748 mini_oidc.cc:152] Starting JWKS server
I20260811 19:00:19.932037 13748 webserver.cc:460] Webserver started at https://127.0.0.1:32863/ using document root <none> and password file <none>
I20260811 19:00:19.932166 13748 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 19:00:19.932456 13748 webserver.cc:460] Webserver started at http://127.0.0.1:38267/ using document root <none> and password file <none>
2026-08-11T19:00:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:00:19Z Disabled control of system clock
I20260811 19:00:19.942729 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786474792496492-13748-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786474792496492-13748-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786474792496492-13748-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.62:45479
--webserver_interface=127.13.109.62
--webserver_port=0
--builtin_ntp_servers=127.13.109.20:37593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.109.62:45479
--enable_jwt_token_auth=true
--jwks_url=https://localhost:32863/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786474792496492-13748-0/testchainca.cert with env {}
W20260811 19:00:20.061225 19386 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:00:20.061530 19386 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:00:20.061578 19386 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:00:20.065687 19386 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:00:20.065763 19386 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:00:20.065796 19386 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:00:20.065815 19386 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:00:20.065830 19386 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 19:00:20.065847 19386 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 19:00:20.065913 19386 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:32863/jwks.json
I20260811 19:00:20.071589 19386 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:37593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786474792496492-13748-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786474792496492-13748-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.109.62:45479
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.62:45479
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:32863/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.109.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786474792496492-13748-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19386
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786474792496492-13748-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:00:20.072799 19386 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:00:20.073895 19386 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:00:20.083061 19393 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:20.083276 19386 server_base.cc:1034] running on GCE node
W20260811 19:00:20.083170 19396 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:20.083061 19392 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:20.084128 19386 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:00:20.085330 19386 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:00:20.086526 19386 hybrid_clock.cc:648] HybridClock initialized: now 1786474820086512 us; error 28 us; skew 500 ppm
I20260811 19:00:20.088454 19386 webserver.cc:460] Webserver started at http://127.13.109.62:33237/ using document root <none> and password file <none>
I20260811 19:00:20.089082 19386 fs_manager.cc:359] Metadata directory not provided
I20260811 19:00:20.089185 19386 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:00:20.089429 19386 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:00:20.091658 19386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786474792496492-13748-0/minicluster-data/master-0/data/instance:
uuid: "56c210b1b76740e6897606aaf8e3ef7b"
format_stamp: "Formatted at 2026-08-11 19:00:20 on dist-test-slave-0kls"
server_key: "eb598432819baae11bbf73cb6323ef94"
server_key_iv: "6e24eb777b46b34f0bcfc81ce722c8b9"
server_key_version: "encryptionkey@0"
I20260811 19:00:20.092373 19386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786474792496492-13748-0/minicluster-data/master-0/wal/instance:
uuid: "56c210b1b76740e6897606aaf8e3ef7b"
format_stamp: "Formatted at 2026-08-11 19:00:20 on dist-test-slave-0kls"
server_key: "eb598432819baae11bbf73cb6323ef94"
server_key_iv: "6e24eb777b46b34f0bcfc81ce722c8b9"
server_key_version: "encryptionkey@0"
I20260811 19:00:20.096181 19386 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 19:00:20.098727 19402 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:20.100015 19386 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 19:00:20.100189 19386 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786474792496492-13748-0/minicluster-data/master-0/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786474792496492-13748-0/minicluster-data/master-0/wal
uuid: "56c210b1b76740e6897606aaf8e3ef7b"
format_stamp: "Formatted at 2026-08-11 19:00:20 on dist-test-slave-0kls"
server_key: "eb598432819baae11bbf73cb6323ef94"
server_key_iv: "6e24eb777b46b34f0bcfc81ce722c8b9"
server_key_version: "encryptionkey@0"
I20260811 19:00:20.100315 19386 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786474792496492-13748-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786474792496492-13748-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786474792496492-13748-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:00:20.115346 19386 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:00:20.116094 19386 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:00:20.116308 19386 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:00:20.123291 19386 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.62:45479
I20260811 19:00:20.123322 19458 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.62:45479 every 8 connection(s)
I20260811 19:00:20.124233 19386 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
I20260811 19:00:20.127554 19459 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:00:20.129225 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 19386
I20260811 19:00:20.129338 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786474792496492-13748-0/minicluster-data/master-0/wal/instance
I20260811 19:00:20.129700 13748 external_mini_cluster.cc:1428] Setting key c173ae18abb180cb319559e14909c5be
I20260811 19:00:20.134467 19459 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56c210b1b76740e6897606aaf8e3ef7b: Bootstrap starting.
I20260811 19:00:20.136861 19459 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 56c210b1b76740e6897606aaf8e3ef7b: Neither blocks nor log segments found. Creating new log.
I20260811 19:00:20.137651 19459 log.cc:824] T 00000000000000000000000000000000 P 56c210b1b76740e6897606aaf8e3ef7b: Log is configured to *not* fsync() on all Append() calls
I20260811 19:00:20.139549 19463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:20.131248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49250 (local address 127.13.109.62:45479)
0811 19:00:20.131972 (+ 724us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:20.131989 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:20.132026 (+ 37us) server_negotiation.cc:407] Connection header received
0811 19:00:20.132807 (+ 781us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:20.132826 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:20.133291 (+ 465us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:20.133627 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:20.134112 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:20.135143 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:20.136079 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:20.136570 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:20.136757 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:20.137680 (+ 923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:20.137685 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:20.137699 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:00:20.138182 (+ 483us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:20.138240 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:00:20.138261 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:20.138378 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:20.138520 (+ 142us) server_negotiation.cc:299] Negotiation successful
0811 19:00:20.138798 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":138,"threads_started":1}
I20260811 19:00:20.140000 19459 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56c210b1b76740e6897606aaf8e3ef7b: No bootstrap required, opened a new log
I20260811 19:00:20.142575 13748 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 19:00:20.144318 19459 raft_consensus.cc:348] T 00000000000000000000000000000000 P 56c210b1b76740e6897606aaf8e3ef7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56c210b1b76740e6897606aaf8e3ef7b" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 45479 } }
I20260811 19:00:20.144541 19459 raft_consensus.cc:374] T 00000000000000000000000000000000 P 56c210b1b76740e6897606aaf8e3ef7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:00:20.144625 19459 raft_consensus.cc:729] T 00000000000000000000000000000000 P 56c210b1b76740e6897606aaf8e3ef7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56c210b1b76740e6897606aaf8e3ef7b, State: Initialized, Role: FOLLOWER
I20260811 19:00:20.145064 19459 consensus_queue.cc:260] T 00000000000000000000000000000000 P 56c210b1b76740e6897606aaf8e3ef7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56c210b1b76740e6897606aaf8e3ef7b" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 45479 } }
I20260811 19:00:20.145216 19459 raft_consensus.cc:388] T 00000000000000000000000000000000 P 56c210b1b76740e6897606aaf8e3ef7b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:00:20.145301 19459 raft_consensus.cc:482] T 00000000000000000000000000000000 P 56c210b1b76740e6897606aaf8e3ef7b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:00:20.145427 19459 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 56c210b1b76740e6897606aaf8e3ef7b [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:00:20.146591 19459 raft_consensus.cc:504] T 00000000000000000000000000000000 P 56c210b1b76740e6897606aaf8e3ef7b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56c210b1b76740e6897606aaf8e3ef7b" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 45479 } }
I20260811 19:00:20.146946 19459 leader_election.cc:302] T 00000000000000000000000000000000 P 56c210b1b76740e6897606aaf8e3ef7b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 56c210b1b76740e6897606aaf8e3ef7b; no voters:
I20260811 19:00:20.147286 19459 leader_election.cc:288] T 00000000000000000000000000000000 P 56c210b1b76740e6897606aaf8e3ef7b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:00:20.147415 19465 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 56c210b1b76740e6897606aaf8e3ef7b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:00:20.147670 19465 raft_consensus.cc:686] T 00000000000000000000000000000000 P 56c210b1b76740e6897606aaf8e3ef7b [term 1 LEADER]: Becoming Leader. State: Replica: 56c210b1b76740e6897606aaf8e3ef7b, State: Running, Role: LEADER
I20260811 19:00:20.148005 19465 consensus_queue.cc:237] T 00000000000000000000000000000000 P 56c210b1b76740e6897606aaf8e3ef7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56c210b1b76740e6897606aaf8e3ef7b" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 45479 } }
I20260811 19:00:20.148589 19459 sys_catalog.cc:564] T 00000000000000000000000000000000 P 56c210b1b76740e6897606aaf8e3ef7b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:00:20.149704 19467 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56c210b1b76740e6897606aaf8e3ef7b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 56c210b1b76740e6897606aaf8e3ef7b. Latest consensus state: current_term: 1 leader_uuid: "56c210b1b76740e6897606aaf8e3ef7b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56c210b1b76740e6897606aaf8e3ef7b" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 45479 } } }
I20260811 19:00:20.149861 19467 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56c210b1b76740e6897606aaf8e3ef7b [sys.catalog]: This master's current role is: LEADER
I20260811 19:00:20.150158 19466 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56c210b1b76740e6897606aaf8e3ef7b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "56c210b1b76740e6897606aaf8e3ef7b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56c210b1b76740e6897606aaf8e3ef7b" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 45479 } } }
I20260811 19:00:20.150264 19466 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56c210b1b76740e6897606aaf8e3ef7b [sys.catalog]: This master's current role is: LEADER
I20260811 19:00:20.150383 19475 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:00:20.154028 19475 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:00:20.159834 19475 catalog_manager.cc:1347] Generated new cluster ID: 4b8799188b024d0b90be4583902c0386
I20260811 19:00:20.159935 19475 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:00:20.169495 19475 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:00:20.170517 19475 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:00:20.178840 19475 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 56c210b1b76740e6897606aaf8e3ef7b: Generated new TSK 0
I20260811 19:00:20.179564 19475 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260811 19:00:20.206796 19492 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260811 19:00:20.206905 19492 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 19:00:20.207713 19463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:20.190664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49260 (local address 127.13.109.62:45479)
0811 19:00:20.190819 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:20.190825 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:20.190955 (+ 130us) server_negotiation.cc:407] Connection header received
0811 19:00:20.191095 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:20.191099 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:20.191165 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:20.191299 (+ 134us) server_negotiation.cc:226] Negotiated authn=JWT
0811 19:00:20.191512 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:20.192367 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:20.193242 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:20.193603 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:20.193719 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:20.194086 (+ 367us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 19:00:20.207229 (+ 13143us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32863/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0811 19:00:20.207551 (+ 322us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:49260: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32863/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":58}
W20260811 19:00:20.207783 19463 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:49260: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32863/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260811 19:00:20.207818 19491 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:00:20.190531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:45479 (local address 127.0.0.1:49260)
0811 19:00:20.190866 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:20.190879 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:20.190977 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:20.191301 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:20.191304 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:20.191309 (+ 5us) client_negotiation.cc:189] Negotiated authn=JWT
0811 19:00:20.191404 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:20.191408 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:20.192483 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:20.192486 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:20.193118 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:20.193126 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:20.193731 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:20.193734 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:20.193851 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:20.193886 (+ 35us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 19:00:20.207469 (+ 13583us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32863/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0811 19:00:20.207642 (+ 173us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.13.109.62:45479: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32863/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":106,"threads_started":1}
I20260811 19:00:20.208267 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 19386
2026-08-11T19:00:20Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (320 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260811 19:00:20.251587 13748 mini_oidc.cc:152] Starting JWKS server
I20260811 19:00:20.252532 13748 webserver.cc:460] Webserver started at https://127.0.0.1:38235/ using document root <none> and password file <none>
I20260811 19:00:20.252650 13748 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 19:00:20.252942 13748 webserver.cc:460] Webserver started at http://127.0.0.1:37205/ using document root <none> and password file <none>
2026-08-11T19:00:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:00:20Z Disabled control of system clock
I20260811 19:00:20.263787 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786474792496492-13748-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786474792496492-13748-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786474792496492-13748-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.62:33495
--webserver_interface=127.13.109.62
--webserver_port=0
--builtin_ntp_servers=127.13.109.20:37347
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.109.62:33495
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38235/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260811 19:00:20.380455 19499 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:00:20.380791 19499 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:00:20.380836 19499 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:00:20.385042 19499 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:00:20.385128 19499 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:00:20.385143 19499 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:00:20.385161 19499 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:00:20.385178 19499 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 19:00:20.385195 19499 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 19:00:20.385210 19499 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:38235/jwks.json
I20260811 19:00:20.390372 19499 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:37347
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786474792496492-13748-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786474792496492-13748-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.109.62:33495
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.62:33495
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38235/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.109.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19499
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786474792496492-13748-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:00:20.391611 19499 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:00:20.393072 19499 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:00:20.402190 19509 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:20.402240 19505 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:20.402190 19507 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:20.402673 19499 server_base.cc:1034] running on GCE node
I20260811 19:00:20.403308 19499 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:00:20.404505 19499 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:00:20.405714 19499 hybrid_clock.cc:648] HybridClock initialized: now 1786474820405696 us; error 38 us; skew 500 ppm
I20260811 19:00:20.407801 19499 webserver.cc:460] Webserver started at http://127.13.109.62:39563/ using document root <none> and password file <none>
I20260811 19:00:20.408465 19499 fs_manager.cc:359] Metadata directory not provided
I20260811 19:00:20.408569 19499 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:00:20.408835 19499 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:00:20.411166 19499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786474792496492-13748-0/minicluster-data/master-0/data/instance:
uuid: "64f00fb487eb447785da35b8611becf5"
format_stamp: "Formatted at 2026-08-11 19:00:20 on dist-test-slave-0kls"
server_key: "1752579b31978aef8384f33666114ebf"
server_key_iv: "57a205a2772b014da3a8f70603d1b725"
server_key_version: "encryptionkey@0"
I20260811 19:00:20.411787 19499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786474792496492-13748-0/minicluster-data/master-0/wal/instance:
uuid: "64f00fb487eb447785da35b8611becf5"
format_stamp: "Formatted at 2026-08-11 19:00:20 on dist-test-slave-0kls"
server_key: "1752579b31978aef8384f33666114ebf"
server_key_iv: "57a205a2772b014da3a8f70603d1b725"
server_key_version: "encryptionkey@0"
I20260811 19:00:20.415831 19499 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 19:00:20.418529 19515 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:20.419924 19499 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 19:00:20.420081 19499 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786474792496492-13748-0/minicluster-data/master-0/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786474792496492-13748-0/minicluster-data/master-0/wal
uuid: "64f00fb487eb447785da35b8611becf5"
format_stamp: "Formatted at 2026-08-11 19:00:20 on dist-test-slave-0kls"
server_key: "1752579b31978aef8384f33666114ebf"
server_key_iv: "57a205a2772b014da3a8f70603d1b725"
server_key_version: "encryptionkey@0"
I20260811 19:00:20.420224 19499 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786474792496492-13748-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786474792496492-13748-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786474792496492-13748-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:00:20.435631 19499 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:00:20.436414 19499 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:00:20.436652 19499 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:00:20.444264 19499 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.62:33495
I20260811 19:00:20.444288 19572 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.62:33495 every 8 connection(s)
I20260811 19:00:20.445312 19499 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
I20260811 19:00:20.448726 19574 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:00:20.450448 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 19499
I20260811 19:00:20.450563 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786474792496492-13748-0/minicluster-data/master-0/wal/instance
I20260811 19:00:20.450873 13748 external_mini_cluster.cc:1428] Setting key 3d787db11bbda0c5a9aed91c4c3b6495
I20260811 19:00:20.456651 19574 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64f00fb487eb447785da35b8611becf5: Bootstrap starting.
I20260811 19:00:20.459673 19574 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 64f00fb487eb447785da35b8611becf5: Neither blocks nor log segments found. Creating new log.
I20260811 19:00:20.460618 19574 log.cc:824] T 00000000000000000000000000000000 P 64f00fb487eb447785da35b8611becf5: Log is configured to *not* fsync() on all Append() calls
I20260811 19:00:20.462630 19577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:20.452608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33140 (local address 127.13.109.62:33495)
0811 19:00:20.453546 (+ 938us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:20.453569 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:20.453619 (+ 50us) server_negotiation.cc:407] Connection header received
0811 19:00:20.454557 (+ 938us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:20.454587 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:20.455165 (+ 578us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:20.455599 (+ 434us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:20.456073 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:20.457530 (+ 1457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:20.458576 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:20.459266 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:20.459442 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:20.460433 (+ 991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:20.460443 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:20.460464 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:00:20.461004 (+ 540us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:20.461103 (+ 99us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:00:20.461134 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:20.461277 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:20.461460 (+ 183us) server_negotiation.cc:299] Negotiation successful
0811 19:00:20.461837 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":467,"thread_start_us":190,"threads_started":1}
I20260811 19:00:20.463074 19574 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64f00fb487eb447785da35b8611becf5: No bootstrap required, opened a new log
I20260811 19:00:20.465596 13748 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 19:00:20.467237 19574 raft_consensus.cc:348] T 00000000000000000000000000000000 P 64f00fb487eb447785da35b8611becf5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64f00fb487eb447785da35b8611becf5" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 33495 } }
I20260811 19:00:20.467600 19574 raft_consensus.cc:374] T 00000000000000000000000000000000 P 64f00fb487eb447785da35b8611becf5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:00:20.467685 19574 raft_consensus.cc:729] T 00000000000000000000000000000000 P 64f00fb487eb447785da35b8611becf5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64f00fb487eb447785da35b8611becf5, State: Initialized, Role: FOLLOWER
I20260811 19:00:20.468139 19574 consensus_queue.cc:260] T 00000000000000000000000000000000 P 64f00fb487eb447785da35b8611becf5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64f00fb487eb447785da35b8611becf5" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 33495 } }
I20260811 19:00:20.468300 19574 raft_consensus.cc:388] T 00000000000000000000000000000000 P 64f00fb487eb447785da35b8611becf5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:00:20.468390 19574 raft_consensus.cc:482] T 00000000000000000000000000000000 P 64f00fb487eb447785da35b8611becf5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:00:20.468518 19574 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 64f00fb487eb447785da35b8611becf5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:00:20.469563 19574 raft_consensus.cc:504] T 00000000000000000000000000000000 P 64f00fb487eb447785da35b8611becf5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64f00fb487eb447785da35b8611becf5" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 33495 } }
I20260811 19:00:20.470000 19574 leader_election.cc:302] T 00000000000000000000000000000000 P 64f00fb487eb447785da35b8611becf5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 64f00fb487eb447785da35b8611becf5; no voters:
I20260811 19:00:20.470355 19574 leader_election.cc:288] T 00000000000000000000000000000000 P 64f00fb487eb447785da35b8611becf5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:00:20.470515 19579 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 64f00fb487eb447785da35b8611becf5 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:00:20.470743 19579 raft_consensus.cc:686] T 00000000000000000000000000000000 P 64f00fb487eb447785da35b8611becf5 [term 1 LEADER]: Becoming Leader. State: Replica: 64f00fb487eb447785da35b8611becf5, State: Running, Role: LEADER
I20260811 19:00:20.471067 19579 consensus_queue.cc:237] T 00000000000000000000000000000000 P 64f00fb487eb447785da35b8611becf5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64f00fb487eb447785da35b8611becf5" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 33495 } }
I20260811 19:00:20.471791 19574 sys_catalog.cc:564] T 00000000000000000000000000000000 P 64f00fb487eb447785da35b8611becf5 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:00:20.473448 19581 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64f00fb487eb447785da35b8611becf5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "64f00fb487eb447785da35b8611becf5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64f00fb487eb447785da35b8611becf5" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 33495 } } }
I20260811 19:00:20.473619 19581 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64f00fb487eb447785da35b8611becf5 [sys.catalog]: This master's current role is: LEADER
I20260811 19:00:20.473948 19582 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64f00fb487eb447785da35b8611becf5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 64f00fb487eb447785da35b8611becf5. Latest consensus state: current_term: 1 leader_uuid: "64f00fb487eb447785da35b8611becf5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64f00fb487eb447785da35b8611becf5" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 33495 } } }
I20260811 19:00:20.474041 19582 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64f00fb487eb447785da35b8611becf5 [sys.catalog]: This master's current role is: LEADER
I20260811 19:00:20.474328 19593 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:00:20.477169 19593 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:00:20.482751 19593 catalog_manager.cc:1347] Generated new cluster ID: 308d85182d4e45e280d930f935b7c313
I20260811 19:00:20.482836 19593 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:00:20.511137 19593 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:00:20.512132 19593 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:00:20.519073 19593 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 64f00fb487eb447785da35b8611becf5: Generated new TSK 0
I20260811 19:00:20.519981 19593 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 19:00:20.544627 19577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:20.539201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33156 (local address 127.13.109.62:33495)
0811 19:00:20.539353 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:20.539358 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:20.539597 (+ 239us) server_negotiation.cc:407] Connection header received
0811 19:00:20.539766 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:20.539770 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:20.539841 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:20.539963 (+ 122us) server_negotiation.cc:226] Negotiated authn=JWT
0811 19:00:20.540189 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:20.541019 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:20.541919 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:20.542278 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:20.542409 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:20.542700 (+ 291us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 19:00:20.544232 (+ 1532us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38235/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0811 19:00:20.544478 (+ 246us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33156: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38235/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":62}
W20260811 19:00:20.544710 19577 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:33156: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38235/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260811 19:00:20.546257 19605 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:00:20.539123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:33495 (local address 127.0.0.1:33156)
0811 19:00:20.539498 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:20.539510 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:20.539613 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:20.539966 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:20.539968 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:20.539975 (+ 7us) client_negotiation.cc:189] Negotiated authn=JWT
0811 19:00:20.540073 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:20.540078 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:20.541128 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:20.541130 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:20.541749 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:20.541755 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:20.542415 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:20.542418 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:20.542480 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:20.542503 (+ 23us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 19:00:20.544408 (+ 1905us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38235/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0811 19:00:20.546078 (+ 1670us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.13.109.62:33495: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38235/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":301,"mutex_wait_us":1470,"thread_start_us":107,"threads_started":1}
I20260811 19:00:20.546849 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 19499
E20260811 19:00:20.559557 19607 webserver.cc:533] Webserver: sslize failed:
E20260811 19:00:20.559646 19607 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-11T19:00:20Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (339 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-11T19:00:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:00:20Z Disabled control of system clock
I20260811 19:00:20.601423 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786474792496492-13748-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786474792496492-13748-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786474792496492-13748-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.62:36331
--webserver_interface=127.13.109.62
--webserver_port=0
--builtin_ntp_servers=127.13.109.20:45627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.109.62:36331
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786474792496492-13748-0/testchainca.cert with env {}
W20260811 19:00:20.728492 19612 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:00:20.728853 19612 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:00:20.728900 19612 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:00:20.733467 19612 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:00:20.733558 19612 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:00:20.733577 19612 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:00:20.733599 19612 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:00:20.733618 19612 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 19:00:20.733639 19612 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 19:00:20.733655 19612 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260811 19:00:20.740082 19612 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:45627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786474792496492-13748-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786474792496492-13748-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.109.62:36331
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.62:36331
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.109.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786474792496492-13748-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19612
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786474792496492-13748-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:00:20.741273 19612 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:00:20.742386 19612 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 19:00:20.752346 19612 server_base.cc:1034] running on GCE node
W20260811 19:00:20.752317 19621 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:20.752372 19619 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:20.752372 19618 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:20.753196 19612 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:00:20.754436 19612 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:00:20.755604 19612 hybrid_clock.cc:648] HybridClock initialized: now 1786474820755583 us; error 47 us; skew 500 ppm
I20260811 19:00:20.757719 19612 webserver.cc:460] Webserver started at http://127.13.109.62:39899/ using document root <none> and password file <none>
I20260811 19:00:20.758492 19612 fs_manager.cc:359] Metadata directory not provided
I20260811 19:00:20.758566 19612 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:00:20.758818 19612 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:00:20.761029 19612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786474792496492-13748-0/minicluster-data/master-0/data/instance:
uuid: "1672dca636cb4523a614917bbab0f595"
format_stamp: "Formatted at 2026-08-11 19:00:20 on dist-test-slave-0kls"
server_key: "fb7f2e26e265dacc1a087289924492ae"
server_key_iv: "6160267aaa796b692eed912b8b7c37c1"
server_key_version: "encryptionkey@0"
I20260811 19:00:20.761639 19612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786474792496492-13748-0/minicluster-data/master-0/wal/instance:
uuid: "1672dca636cb4523a614917bbab0f595"
format_stamp: "Formatted at 2026-08-11 19:00:20 on dist-test-slave-0kls"
server_key: "fb7f2e26e265dacc1a087289924492ae"
server_key_iv: "6160267aaa796b692eed912b8b7c37c1"
server_key_version: "encryptionkey@0"
I20260811 19:00:20.765769 19612 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260811 19:00:20.768735 19627 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:20.770254 19612 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 19:00:20.770447 19612 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786474792496492-13748-0/minicluster-data/master-0/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786474792496492-13748-0/minicluster-data/master-0/wal
uuid: "1672dca636cb4523a614917bbab0f595"
format_stamp: "Formatted at 2026-08-11 19:00:20 on dist-test-slave-0kls"
server_key: "fb7f2e26e265dacc1a087289924492ae"
server_key_iv: "6160267aaa796b692eed912b8b7c37c1"
server_key_version: "encryptionkey@0"
I20260811 19:00:20.770596 19612 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786474792496492-13748-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786474792496492-13748-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786474792496492-13748-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:00:20.797537 19612 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:00:20.798378 19612 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:00:20.798604 19612 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:00:20.806195 19612 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.62:36331
I20260811 19:00:20.806205 19681 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.62:36331 every 8 connection(s)
I20260811 19:00:20.807279 19612 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
I20260811 19:00:20.809348 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 19612
I20260811 19:00:20.809451 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786474792496492-13748-0/minicluster-data/master-0/wal/instance
I20260811 19:00:20.809775 13748 external_mini_cluster.cc:1428] Setting key d155040cc84ff0e6302258a3b86eb884
I20260811 19:00:20.810918 19683 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:00:20.818308 19683 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1672dca636cb4523a614917bbab0f595: Bootstrap starting.
I20260811 19:00:20.820956 19687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:20.811385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41060 (local address 127.13.109.62:36331)
0811 19:00:20.812105 (+ 720us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:20.812127 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:20.812174 (+ 47us) server_negotiation.cc:407] Connection header received
0811 19:00:20.813098 (+ 924us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:20.813124 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:20.813673 (+ 549us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:20.814093 (+ 420us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:20.814567 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:20.816008 (+ 1441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:20.816871 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:20.817627 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:20.817848 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:20.818753 (+ 905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:20.818759 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:20.818775 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:00:20.819282 (+ 507us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:20.819354 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:00:20.819382 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:20.819509 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:20.819635 (+ 126us) server_negotiation.cc:299] Negotiation successful
0811 19:00:20.819962 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":134,"threads_started":1}
I20260811 19:00:20.821327 19683 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1672dca636cb4523a614917bbab0f595: Neither blocks nor log segments found. Creating new log.
I20260811 19:00:20.822283 19683 log.cc:824] T 00000000000000000000000000000000 P 1672dca636cb4523a614917bbab0f595: Log is configured to *not* fsync() on all Append() calls
I20260811 19:00:20.824288 13748 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 19:00:20.824699 19683 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1672dca636cb4523a614917bbab0f595: No bootstrap required, opened a new log
I20260811 19:00:20.828675 19683 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1672dca636cb4523a614917bbab0f595 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1672dca636cb4523a614917bbab0f595" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 36331 } }
I20260811 19:00:20.828953 19683 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1672dca636cb4523a614917bbab0f595 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:00:20.828994 19683 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1672dca636cb4523a614917bbab0f595 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1672dca636cb4523a614917bbab0f595, State: Initialized, Role: FOLLOWER
I20260811 19:00:20.829489 19683 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1672dca636cb4523a614917bbab0f595 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1672dca636cb4523a614917bbab0f595" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 36331 } }
I20260811 19:00:20.829651 19683 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1672dca636cb4523a614917bbab0f595 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:00:20.829751 19683 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1672dca636cb4523a614917bbab0f595 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:00:20.829932 19683 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1672dca636cb4523a614917bbab0f595 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:00:20.831058 19683 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1672dca636cb4523a614917bbab0f595 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1672dca636cb4523a614917bbab0f595" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 36331 } }
I20260811 19:00:20.831472 19683 leader_election.cc:302] T 00000000000000000000000000000000 P 1672dca636cb4523a614917bbab0f595 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1672dca636cb4523a614917bbab0f595; no voters:
I20260811 19:00:20.831866 19683 leader_election.cc:288] T 00000000000000000000000000000000 P 1672dca636cb4523a614917bbab0f595 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:00:20.831979 19689 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1672dca636cb4523a614917bbab0f595 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:00:20.832175 19689 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1672dca636cb4523a614917bbab0f595 [term 1 LEADER]: Becoming Leader. State: Replica: 1672dca636cb4523a614917bbab0f595, State: Running, Role: LEADER
I20260811 19:00:20.832500 19689 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1672dca636cb4523a614917bbab0f595 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1672dca636cb4523a614917bbab0f595" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 36331 } }
I20260811 19:00:20.833344 19683 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1672dca636cb4523a614917bbab0f595 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:00:20.833837 19689 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1672dca636cb4523a614917bbab0f595 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1672dca636cb4523a614917bbab0f595. Latest consensus state: current_term: 1 leader_uuid: "1672dca636cb4523a614917bbab0f595" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1672dca636cb4523a614917bbab0f595" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 36331 } } }
I20260811 19:00:20.833962 19689 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1672dca636cb4523a614917bbab0f595 [sys.catalog]: This master's current role is: LEADER
I20260811 19:00:20.834381 19693 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:00:20.834401 19690 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1672dca636cb4523a614917bbab0f595 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1672dca636cb4523a614917bbab0f595" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1672dca636cb4523a614917bbab0f595" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 36331 } } }
I20260811 19:00:20.834661 19690 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1672dca636cb4523a614917bbab0f595 [sys.catalog]: This master's current role is: LEADER
I20260811 19:00:20.838042 19693 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:00:20.844908 19693 catalog_manager.cc:1347] Generated new cluster ID: 8e727b2fb26e4e9abcdbab8fcb9f04d7
I20260811 19:00:20.845044 19693 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:00:20.854324 19693 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:00:20.855309 19693 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:00:20.861078 19693 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1672dca636cb4523a614917bbab0f595: Generated new TSK 0
I20260811 19:00:20.861900 19693 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 19:00:21.400777 19687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:20.871416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41072 (local address 127.13.109.62:36331)
0811 19:00:20.871555 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:20.871560 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:20.871727 (+ 167us) server_negotiation.cc:407] Connection header received
0811 19:00:20.871869 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:20.871872 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:20.871929 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:20.872051 (+ 122us) server_negotiation.cc:226] Negotiated authn=JWT
0811 19:00:20.872265 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:20.873040 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:20.873995 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:20.874342 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:20.874449 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:20.874826 (+ 377us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 19:00:21.400342 (+525516us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0811 19:00:21.400646 (+ 304us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41072: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":52}
W20260811 19:00:21.400858 19687 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:41072: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260811 19:00:21.401194 19716 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:00:20.871309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:36331 (local address 127.0.0.1:41072)
0811 19:00:20.871639 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:20.871653 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:20.871754 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:20.872056 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:20.872058 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:20.872064 (+ 6us) client_negotiation.cc:189] Negotiated authn=JWT
0811 19:00:20.872164 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:20.872168 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:20.873175 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:20.873177 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:20.873885 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:20.873894 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:20.874487 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:20.874491 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:20.874615 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:20.874643 (+ 28us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 19:00:21.400649 (+526006us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0811 19:00:21.400892 (+ 243us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.13.109.62:36331: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":115,"threads_started":1}
I20260811 19:00:21.401917 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 19612
2026-08-11T19:00:21Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (834 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (134 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (133 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:00:21 dist-test-slave-0kls krb5kdc[19728](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:00:21 dist-test-slave-0kls krb5kdc[19728](info): set up 2 sockets
Aug 11 19:00:21 dist-test-slave-0kls krb5kdc[19728](info): commencing operation
krb5kdc: starting...
W20260811 19:00:22.769290 13748 mini_kdc.cc:121] Time spent starting KDC: real 1.075s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/krb5kdc/test-user.keytab.
Aug 11 19:00:22 dist-test-slave-0kls krb5kdc[19728](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474822, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:00:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:00:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.109.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.109.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.109.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab.
Entry for principal kudu/127.13.109.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab.
I20260811 19:00:22.909916 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.62:42097
--webserver_interface=127.13.109.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab
--principal=kudu/127.13.109.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:44079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.109.62:42097
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:00:23.029942 19741 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:00:23.030328 19741 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:00:23.030417 19741 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:00:23.034818 19741 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:00:23.034941 19741 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:00:23.034977 19741 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:00:23.035020 19741 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:00:23.035112 19741 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:00:23.040973 19741 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:44079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.109.62:42097
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab
--principal=kudu/127.13.109.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.62:42097
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.109.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19741
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:00:23.042351 19741 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:00:23.043491 19741 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:00:23.052700 19746 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:23.052695 19750 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:23.052695 19748 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:23.052985 19741 server_base.cc:1034] running on GCE node
I20260811 19:00:23.053778 19741 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:00:23.055052 19741 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:00:23.056257 19741 hybrid_clock.cc:648] HybridClock initialized: now 1786474823056234 us; error 33 us; skew 500 ppm
Aug 11 19:00:23 dist-test-slave-0kls krb5kdc[19728](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474823, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:00:23.063004 19741 init.cc:377] Logged in from keytab as kudu/127.13.109.62@KRBTEST.COM (short username kudu)
I20260811 19:00:23.064105 19741 webserver.cc:460] Webserver started at http://127.13.109.62:42379/ using document root <none> and password file <none>
I20260811 19:00:23.064751 19741 fs_manager.cc:359] Metadata directory not provided
I20260811 19:00:23.064857 19741 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:00:23.065124 19741 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:00:23.067458 19741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/minicluster-data/master-0/data/instance:
uuid: "67c0b8b8604f4641a567ff8346fe0542"
format_stamp: "Formatted at 2026-08-11 19:00:23 on dist-test-slave-0kls"
server_key: "fbd6cb0361b5e89e45db5fc5f16014fb"
server_key_iv: "db359b9f7c5d275b7595e8cae39f31c6"
server_key_version: "encryptionkey@0"
I20260811 19:00:23.068075 19741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/minicluster-data/master-0/wal/instance:
uuid: "67c0b8b8604f4641a567ff8346fe0542"
format_stamp: "Formatted at 2026-08-11 19:00:23 on dist-test-slave-0kls"
server_key: "fbd6cb0361b5e89e45db5fc5f16014fb"
server_key_iv: "db359b9f7c5d275b7595e8cae39f31c6"
server_key_version: "encryptionkey@0"
I20260811 19:00:23.072117 19741 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 19:00:23.074960 19757 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:23.076359 19741 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 19:00:23.076529 19741 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/minicluster-data/master-0/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/minicluster-data/master-0/wal
uuid: "67c0b8b8604f4641a567ff8346fe0542"
format_stamp: "Formatted at 2026-08-11 19:00:23 on dist-test-slave-0kls"
server_key: "fbd6cb0361b5e89e45db5fc5f16014fb"
server_key_iv: "db359b9f7c5d275b7595e8cae39f31c6"
server_key_version: "encryptionkey@0"
I20260811 19:00:23.076664 19741 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:00:23.103260 19741 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:00:23.107048 19741 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:00:23.107287 19741 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:00:23.115371 19741 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.62:42097
I20260811 19:00:23.115397 19812 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.62:42097 every 8 connection(s)
I20260811 19:00:23.116456 19741 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
I20260811 19:00:23.118542 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 19741
I20260811 19:00:23.118703 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/minicluster-data/master-0/wal/instance
I20260811 19:00:23.119043 13748 external_mini_cluster.cc:1428] Setting key d1fce1294b9fc2b46ff175efdb4a3ed1
I20260811 19:00:23.120004 19813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:00:23.126343 19813 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67c0b8b8604f4641a567ff8346fe0542: Bootstrap starting.
Aug 11 19:00:23 dist-test-slave-0kls krb5kdc[19728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474822, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
I20260811 19:00:23.129206 19813 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 67c0b8b8604f4641a567ff8346fe0542: Neither blocks nor log segments found. Creating new log.
I20260811 19:00:23.130108 19813 log.cc:824] T 00000000000000000000000000000000 P 67c0b8b8604f4641a567ff8346fe0542: Log is configured to *not* fsync() on all Append() calls
I20260811 19:00:23.132666 19813 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67c0b8b8604f4641a567ff8346fe0542: No bootstrap required, opened a new log
I20260811 19:00:23.134706 19816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:23.120669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54014 (local address 127.13.109.62:42097)
0811 19:00:23.121177 (+ 508us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:23.121193 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:23.121231 (+ 38us) server_negotiation.cc:407] Connection header received
0811 19:00:23.121970 (+ 739us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:23.121989 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:23.122335 (+ 346us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:23.122680 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:23.123529 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:23.124522 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:23.125355 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:23.125880 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:23.126047 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:23.129057 (+ 3010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:23.129062 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:23.129076 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:23.129107 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:23.131340 (+ 2233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:23.131895 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:23.131900 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:23.131905 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:23.132016 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:23.132400 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:23.132403 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:23.132404 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:23.132947 (+ 543us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:23.133132 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:23.133538 (+ 406us) server_negotiation.cc:299] Negotiation successful
0811 19:00:23.133920 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":170,"threads_started":1}
I20260811 19:00:23.137122 19813 raft_consensus.cc:348] T 00000000000000000000000000000000 P 67c0b8b8604f4641a567ff8346fe0542 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67c0b8b8604f4641a567ff8346fe0542" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 42097 } }
I20260811 19:00:23.137389 19813 raft_consensus.cc:374] T 00000000000000000000000000000000 P 67c0b8b8604f4641a567ff8346fe0542 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:00:23.137478 19813 raft_consensus.cc:729] T 00000000000000000000000000000000 P 67c0b8b8604f4641a567ff8346fe0542 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67c0b8b8604f4641a567ff8346fe0542, State: Initialized, Role: FOLLOWER
I20260811 19:00:23.138005 19813 consensus_queue.cc:260] T 00000000000000000000000000000000 P 67c0b8b8604f4641a567ff8346fe0542 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67c0b8b8604f4641a567ff8346fe0542" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 42097 } }
I20260811 19:00:23.138190 19813 raft_consensus.cc:388] T 00000000000000000000000000000000 P 67c0b8b8604f4641a567ff8346fe0542 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:00:23.138276 19813 raft_consensus.cc:482] T 00000000000000000000000000000000 P 67c0b8b8604f4641a567ff8346fe0542 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:00:23.138415 19813 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 67c0b8b8604f4641a567ff8346fe0542 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:00:23.139448 19813 raft_consensus.cc:504] T 00000000000000000000000000000000 P 67c0b8b8604f4641a567ff8346fe0542 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67c0b8b8604f4641a567ff8346fe0542" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 42097 } }
I20260811 19:00:23.139843 19813 leader_election.cc:302] T 00000000000000000000000000000000 P 67c0b8b8604f4641a567ff8346fe0542 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 67c0b8b8604f4641a567ff8346fe0542; no voters:
I20260811 19:00:23.140225 19813 leader_election.cc:288] T 00000000000000000000000000000000 P 67c0b8b8604f4641a567ff8346fe0542 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:00:23.140477 19818 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 67c0b8b8604f4641a567ff8346fe0542 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:00:23.140720 19818 raft_consensus.cc:686] T 00000000000000000000000000000000 P 67c0b8b8604f4641a567ff8346fe0542 [term 1 LEADER]: Becoming Leader. State: Replica: 67c0b8b8604f4641a567ff8346fe0542, State: Running, Role: LEADER
I20260811 19:00:23.141110 19818 consensus_queue.cc:237] T 00000000000000000000000000000000 P 67c0b8b8604f4641a567ff8346fe0542 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67c0b8b8604f4641a567ff8346fe0542" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 42097 } }
I20260811 19:00:23.141722 19813 sys_catalog.cc:564] T 00000000000000000000000000000000 P 67c0b8b8604f4641a567ff8346fe0542 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:00:23.142450 19820 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67c0b8b8604f4641a567ff8346fe0542 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 67c0b8b8604f4641a567ff8346fe0542. Latest consensus state: current_term: 1 leader_uuid: "67c0b8b8604f4641a567ff8346fe0542" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67c0b8b8604f4641a567ff8346fe0542" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 42097 } } }
I20260811 19:00:23.142584 19820 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67c0b8b8604f4641a567ff8346fe0542 [sys.catalog]: This master's current role is: LEADER
I20260811 19:00:23.142918 19821 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67c0b8b8604f4641a567ff8346fe0542 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "67c0b8b8604f4641a567ff8346fe0542" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67c0b8b8604f4641a567ff8346fe0542" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 42097 } } }
I20260811 19:00:23.143046 19821 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67c0b8b8604f4641a567ff8346fe0542 [sys.catalog]: This master's current role is: LEADER
I20260811 19:00:23.143045 19825 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:00:23.146456 19825 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:00:23.152732 19825 catalog_manager.cc:1347] Generated new cluster ID: 99a0cde843c0414dbecdd0ef4e9a34af
I20260811 19:00:23.152825 19825 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:00:23.161410 19825 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:00:23.162461 19825 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:00:23.171095 19825 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 67c0b8b8604f4641a567ff8346fe0542: Generated new TSK 0
I20260811 19:00:23.171895 19825 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.109.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.109.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.109.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.1.keytab.
Entry for principal kudu/127.13.109.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.1.keytab.
I20260811 19:00:23.205161 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.1:0
--local_ip_for_outbound_sockets=127.13.109.1
--webserver_interface=127.13.109.1
--webserver_port=0
--tserver_master_addrs=127.13.109.62:42097
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.1.keytab
--principal=kudu/127.13.109.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:44079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:00:23.325184 19845 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:00:23.325467 19845 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:00:23.325511 19845 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:00:23.329627 19845 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:00:23.329694 19845 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:00:23.329753 19845 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.1
I20260811 19:00:23.335321 19845 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:44079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.1.keytab
--principal=kudu/127.13.109.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.109.1
--webserver_port=0
--tserver_master_addrs=127.13.109.62:42097
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19845
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.1
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:00:23.336434 19845 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:00:23.337569 19845 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:00:23.347927 19851 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:23.348019 19845 server_base.cc:1034] running on GCE node
W20260811 19:00:23.347932 19854 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:23.348215 19852 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:23.348672 19845 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:00:23.349342 19845 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:00:23.350544 19845 hybrid_clock.cc:648] HybridClock initialized: now 1786474823350533 us; error 26 us; skew 500 ppm
Aug 11 19:00:23 dist-test-slave-0kls krb5kdc[19728](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474823, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:00:23.353406 19845 init.cc:377] Logged in from keytab as kudu/127.13.109.1@KRBTEST.COM (short username kudu)
I20260811 19:00:23.354442 19845 webserver.cc:460] Webserver started at http://127.13.109.1:43351/ using document root <none> and password file <none>
I20260811 19:00:23.354986 19845 fs_manager.cc:359] Metadata directory not provided
I20260811 19:00:23.355047 19845 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:00:23.355217 19845 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:00:23.357221 19845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/minicluster-data/ts-0/data/instance:
uuid: "4f8dac4064fe4daaa83b6acc079f08b2"
format_stamp: "Formatted at 2026-08-11 19:00:23 on dist-test-slave-0kls"
server_key: "4928d0e0f391b53b1b8bf37f6a683ebc"
server_key_iv: "2862743ba709f78360344211a2122354"
server_key_version: "encryptionkey@0"
I20260811 19:00:23.357743 19845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/minicluster-data/ts-0/wal/instance:
uuid: "4f8dac4064fe4daaa83b6acc079f08b2"
format_stamp: "Formatted at 2026-08-11 19:00:23 on dist-test-slave-0kls"
server_key: "4928d0e0f391b53b1b8bf37f6a683ebc"
server_key_iv: "2862743ba709f78360344211a2122354"
server_key_version: "encryptionkey@0"
I20260811 19:00:23.361943 19845 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 19:00:23.364558 19861 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:23.365892 19845 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 19:00:23.366014 19845 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/minicluster-data/ts-0/wal
uuid: "4f8dac4064fe4daaa83b6acc079f08b2"
format_stamp: "Formatted at 2026-08-11 19:00:23 on dist-test-slave-0kls"
server_key: "4928d0e0f391b53b1b8bf37f6a683ebc"
server_key_iv: "2862743ba709f78360344211a2122354"
server_key_version: "encryptionkey@0"
I20260811 19:00:23.366130 19845 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:00:23.378564 19845 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:00:23.382179 19845 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:00:23.382390 19845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:00:23.383105 19845 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:00:23.384001 19845 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:00:23.384052 19845 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:23.384123 19845 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:00:23.384156 19845 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:23.394467 19845 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.1:37597
I20260811 19:00:23.394488 19984 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.1:37597 every 8 connection(s)
I20260811 19:00:23.395440 19845 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
I20260811 19:00:23.403225 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 19845
I20260811 19:00:23.403352 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/minicluster-data/ts-0/wal/instance
I20260811 19:00:23.403707 13748 external_mini_cluster.cc:1428] Setting key 6302facad9bb9f1131a1d95540421496
Aug 11 19:00:23 dist-test-slave-0kls krb5kdc[19728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474823, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.1@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
I20260811 19:00:23.410641 19816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:23.397487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.1:42145 (local address 127.13.109.62:42097)
0811 19:00:23.397689 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:23.397692 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:23.398626 (+ 934us) server_negotiation.cc:407] Connection header received
0811 19:00:23.400086 (+ 1460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:23.400090 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:23.400156 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:23.400263 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:23.401578 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:23.402198 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:23.403493 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:23.403744 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:23.403864 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:23.406782 (+ 2918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:23.406786 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:23.406789 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:23.406822 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:23.408395 (+ 1573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:23.408940 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:23.408944 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:23.408946 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:23.409035 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:23.409392 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:23.409396 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:23.409398 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:23.409604 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:23.409711 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:23.410315 (+ 604us) server_negotiation.cc:299] Negotiation successful
0811 19:00:23.410459 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260811 19:00:23.411501 19987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:23.397952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:42097 (local address 127.13.109.1:42145)
0811 19:00:23.398466 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:23.398512 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:23.399847 (+ 1335us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:23.400458 (+ 611us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:23.400465 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:23.400858 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:23.401397 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:23.401411 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:23.402357 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:23.402360 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:23.403322 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:23.403339 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:23.403933 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:23.403937 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:23.404164 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:23.404881 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:23.404900 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:23.406605 (+ 1705us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:00:23.408524 (+ 1919us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:23.408531 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:23.408548 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:23.408829 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:23.409134 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:23.409138 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:23.409140 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:23.409291 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:23.409726 (+ 435us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:00:23.409735 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:00:23.410062 (+ 327us) client_negotiation.cc:769] Sending connection context
0811 19:00:23.410273 (+ 211us) client_negotiation.cc:240] Negotiation successful
0811 19:00:23.410625 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":155,"threads_started":1}
I20260811 19:00:23.412791 19985 heartbeater.cc:347] Connected to a master server at 127.13.109.62:42097
I20260811 19:00:23.413059 19985 heartbeater.cc:464] Registering TS with master...
I20260811 19:00:23.413626 19985 heartbeater.cc:511] Master 127.13.109.62:42097 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.13.109.2@KRBTEST.COM; defaulting to no policy
I20260811 19:00:23.415266 19774 ts_manager.cc:194] Registered new tserver with Master: 4f8dac4064fe4daaa83b6acc079f08b2 (127.13.109.1:37597)
I20260811 19:00:23.416465 19774 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.109.1@KRBTEST.COM'} at 127.13.109.1:42145
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.109.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.109.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.2.keytab.
Entry for principal kudu/127.13.109.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.2.keytab.
I20260811 19:00:23.438234 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.2:0
--local_ip_for_outbound_sockets=127.13.109.2
--webserver_interface=127.13.109.2
--webserver_port=0
--tserver_master_addrs=127.13.109.62:42097
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.2.keytab
--principal=kudu/127.13.109.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:44079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:00:23.564572 19990 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:00:23.564954 19990 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:00:23.565037 19990 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:00:23.569306 19990 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:00:23.569373 19990 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:00:23.569468 19990 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.2
I20260811 19:00:23.575316 19990 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:44079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.2.keytab
--principal=kudu/127.13.109.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.109.2
--webserver_port=0
--tserver_master_addrs=127.13.109.62:42097
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19990
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.2
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:00:23.576542 19990 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:00:23.577623 19990 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:00:23.588454 19996 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:23.588438 19995 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:23.588981 19999 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:23.589195 19990 server_base.cc:1034] running on GCE node
I20260811 19:00:23.589773 19990 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:00:23.590598 19990 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:00:23.591794 19990 hybrid_clock.cc:648] HybridClock initialized: now 1786474823591775 us; error 42 us; skew 500 ppm
Aug 11 19:00:23 dist-test-slave-0kls krb5kdc[19728](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474823, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:00:23.595186 19990 init.cc:377] Logged in from keytab as kudu/127.13.109.2@KRBTEST.COM (short username kudu)
I20260811 19:00:23.596371 19990 webserver.cc:460] Webserver started at http://127.13.109.2:44103/ using document root <none> and password file <none>
I20260811 19:00:23.597029 19990 fs_manager.cc:359] Metadata directory not provided
I20260811 19:00:23.597131 19990 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:00:23.597393 19990 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:00:23.599794 19990 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/minicluster-data/ts-1/data/instance:
uuid: "5bbd2af7214d421d9a85dcfca42027ce"
format_stamp: "Formatted at 2026-08-11 19:00:23 on dist-test-slave-0kls"
server_key: "eb579d9df187fa6f2bb880348cebd892"
server_key_iv: "60184d33e1ffe51566c87720051aba8c"
server_key_version: "encryptionkey@0"
I20260811 19:00:23.600433 19990 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/minicluster-data/ts-1/wal/instance:
uuid: "5bbd2af7214d421d9a85dcfca42027ce"
format_stamp: "Formatted at 2026-08-11 19:00:23 on dist-test-slave-0kls"
server_key: "eb579d9df187fa6f2bb880348cebd892"
server_key_iv: "60184d33e1ffe51566c87720051aba8c"
server_key_version: "encryptionkey@0"
I20260811 19:00:23.604787 19990 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 19:00:23.607666 20006 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:23.609464 19990 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 19:00:23.609632 19990 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/minicluster-data/ts-1/wal
uuid: "5bbd2af7214d421d9a85dcfca42027ce"
format_stamp: "Formatted at 2026-08-11 19:00:23 on dist-test-slave-0kls"
server_key: "eb579d9df187fa6f2bb880348cebd892"
server_key_iv: "60184d33e1ffe51566c87720051aba8c"
server_key_version: "encryptionkey@0"
I20260811 19:00:23.609813 19990 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:00:23.633637 19990 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:00:23.637380 19990 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:00:23.637571 19990 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:00:23.638321 19990 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:00:23.639444 19990 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:00:23.639518 19990 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:23.639566 19990 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:00:23.639588 19990 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:23.649344 19990 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.2:41961
I20260811 19:00:23.649453 20128 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.2:41961 every 8 connection(s)
I20260811 19:00:23.650521 19990 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
I20260811 19:00:23.656730 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 19990
I20260811 19:00:23.656859 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/minicluster-data/ts-1/wal/instance
I20260811 19:00:23.657191 13748 external_mini_cluster.cc:1428] Setting key c17db7b7dbadd0450192aa1ea6c1f2b8
Aug 11 19:00:23 dist-test-slave-0kls krb5kdc[19728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474823, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.2@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
I20260811 19:00:23.666507 19816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:23.652627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.2:48417 (local address 127.13.109.62:42097)
0811 19:00:23.652791 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:23.652797 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:23.653858 (+ 1061us) server_negotiation.cc:407] Connection header received
0811 19:00:23.655240 (+ 1382us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:23.655243 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:23.655295 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:23.655373 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:23.656683 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:23.657353 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:23.658877 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:23.659110 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:23.659195 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:23.662643 (+ 3448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:23.662647 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:23.662650 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:23.662677 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:23.664115 (+ 1438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:23.664724 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:23.664728 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:23.664730 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:23.664818 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:23.665200 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:23.665211 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:23.665212 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:23.665417 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:23.665529 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:23.666176 (+ 647us) server_negotiation.cc:299] Negotiation successful
0811 19:00:23.666329 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 19:00:23.667449 20131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:23.653116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:42097 (local address 127.13.109.2:48417)
0811 19:00:23.653629 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:23.653677 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:23.655019 (+ 1342us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:23.655565 (+ 546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:23.655572 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:23.655953 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:23.656506 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:23.656530 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:23.657493 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:23.657497 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:23.658754 (+ 1257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:23.658767 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:23.659291 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:23.659299 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:23.659556 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:23.660238 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:23.660264 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:23.662380 (+ 2116us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:00:23.664291 (+ 1911us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:23.664298 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:23.664319 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:23.664603 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:23.664921 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:23.664925 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:23.664927 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:23.665087 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:23.665529 (+ 442us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:00:23.665537 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:00:23.665906 (+ 369us) client_negotiation.cc:769] Sending connection context
0811 19:00:23.666113 (+ 207us) client_negotiation.cc:240] Negotiation successful
0811 19:00:23.666495 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":129,"threads_started":1}
WARNING: no policy specified for kudu/127.13.109.3@KRBTEST.COM; defaulting to no policy
I20260811 19:00:23.668804 20129 heartbeater.cc:347] Connected to a master server at 127.13.109.62:42097
I20260811 19:00:23.669082 20129 heartbeater.cc:464] Registering TS with master...
I20260811 19:00:23.669858 20129 heartbeater.cc:511] Master 127.13.109.62:42097 requested a full tablet report, sending...
I20260811 19:00:23.671129 19774 ts_manager.cc:194] Registered new tserver with Master: 5bbd2af7214d421d9a85dcfca42027ce (127.13.109.2:41961)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.109.3@KRBTEST.COM" created.
I20260811 19:00:23.672027 19774 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.109.2@KRBTEST.COM'} at 127.13.109.2:48417
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.109.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.3.keytab.
Entry for principal kudu/127.13.109.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.3.keytab.
I20260811 19:00:23.689677 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.3:0
--local_ip_for_outbound_sockets=127.13.109.3
--webserver_interface=127.13.109.3
--webserver_port=0
--tserver_master_addrs=127.13.109.62:42097
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.3.keytab
--principal=kudu/127.13.109.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:44079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:00:23.813318 20134 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:00:23.813630 20134 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:00:23.813676 20134 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:00:23.818014 20134 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:00:23.818100 20134 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:00:23.818166 20134 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.3
I20260811 19:00:23.823942 20134 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:44079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.3.keytab
--principal=kudu/127.13.109.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.109.3
--webserver_port=0
--tserver_master_addrs=127.13.109.62:42097
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20134
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.3
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:00:23.825191 20134 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:00:23.826309 20134 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:00:23.837200 20142 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:23.837137 20140 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:23.837144 20139 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:23.837541 20134 server_base.cc:1034] running on GCE node
I20260811 19:00:23.838299 20134 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:00:23.839201 20134 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:00:23.840416 20134 hybrid_clock.cc:648] HybridClock initialized: now 1786474823840371 us; error 62 us; skew 500 ppm
Aug 11 19:00:23 dist-test-slave-0kls krb5kdc[19728](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474823, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:00:23.843930 20134 init.cc:377] Logged in from keytab as kudu/127.13.109.3@KRBTEST.COM (short username kudu)
I20260811 19:00:23.845031 20134 webserver.cc:460] Webserver started at http://127.13.109.3:46505/ using document root <none> and password file <none>
I20260811 19:00:23.845692 20134 fs_manager.cc:359] Metadata directory not provided
I20260811 19:00:23.845830 20134 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:00:23.846112 20134 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:00:23.848652 20134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/minicluster-data/ts-2/data/instance:
uuid: "5944cc29b3004a0c968623e7aef7611b"
format_stamp: "Formatted at 2026-08-11 19:00:23 on dist-test-slave-0kls"
server_key: "71ce0452ea0f1da90b21914b1f12306b"
server_key_iv: "cfd84193f7cd112e150a11838025857d"
server_key_version: "encryptionkey@0"
I20260811 19:00:23.849313 20134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/minicluster-data/ts-2/wal/instance:
uuid: "5944cc29b3004a0c968623e7aef7611b"
format_stamp: "Formatted at 2026-08-11 19:00:23 on dist-test-slave-0kls"
server_key: "71ce0452ea0f1da90b21914b1f12306b"
server_key_iv: "cfd84193f7cd112e150a11838025857d"
server_key_version: "encryptionkey@0"
I20260811 19:00:23.853471 20134 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260811 19:00:23.856259 20150 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:23.857558 20134 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 19:00:23.857704 20134 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/minicluster-data/ts-2/wal
uuid: "5944cc29b3004a0c968623e7aef7611b"
format_stamp: "Formatted at 2026-08-11 19:00:23 on dist-test-slave-0kls"
server_key: "71ce0452ea0f1da90b21914b1f12306b"
server_key_iv: "cfd84193f7cd112e150a11838025857d"
server_key_version: "encryptionkey@0"
I20260811 19:00:23.857870 20134 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:00:23.903503 20134 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:00:23.906868 20134 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:00:23.907133 20134 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:00:23.908118 20134 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:00:23.909101 20134 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:00:23.909178 20134 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:23.909258 20134 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:00:23.909291 20134 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:23.918846 20134 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.3:34847
I20260811 19:00:23.918911 20271 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.3:34847 every 8 connection(s)
I20260811 19:00:23.919899 20134 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
I20260811 19:00:23.929114 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 20134
I20260811 19:00:23.929313 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/minicluster-data/ts-2/wal/instance
I20260811 19:00:23.929664 13748 external_mini_cluster.cc:1428] Setting key 5be42e78c0253783210bbb6135381a41
Aug 11 19:00:23 dist-test-slave-0kls krb5kdc[19728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474823, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.3@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
I20260811 19:00:23.935315 19816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:23.922174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.3:41081 (local address 127.13.109.62:42097)
0811 19:00:23.922327 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:23.922331 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:23.923305 (+ 974us) server_negotiation.cc:407] Connection header received
0811 19:00:23.924753 (+ 1448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:23.924757 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:23.924822 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:23.924935 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:23.926239 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:23.926784 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:23.928222 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:23.928451 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:23.928577 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:23.931823 (+ 3246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:23.931826 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:23.931828 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:23.931855 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:23.933264 (+ 1409us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:23.933753 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:23.933756 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:23.933757 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:23.933907 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:23.934312 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:23.934315 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:23.934316 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:23.934504 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:23.934591 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:23.935085 (+ 494us) server_negotiation.cc:299] Negotiation successful
0811 19:00:23.935195 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 19:00:23.936265 20274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:23.922589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:42097 (local address 127.13.109.3:41081)
0811 19:00:23.923128 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:23.923174 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:23.924507 (+ 1333us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:23.925135 (+ 628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:23.925143 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:23.925533 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:23.926080 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:23.926094 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:23.926923 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:23.926931 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:23.928098 (+ 1167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:23.928110 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:23.928629 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:23.928633 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:23.928869 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:23.929704 (+ 835us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:23.929722 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:23.931623 (+ 1901us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:00:23.933374 (+ 1751us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:23.933379 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:23.933392 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:23.933659 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:23.934023 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:23.934028 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:23.934030 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:23.934215 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:23.934610 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:00:23.934615 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:00:23.934866 (+ 251us) client_negotiation.cc:769] Sending connection context
0811 19:00:23.935033 (+ 167us) client_negotiation.cc:240] Negotiation successful
0811 19:00:23.935328 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":136,"threads_started":1}
I20260811 19:00:23.937481 20272 heartbeater.cc:347] Connected to a master server at 127.13.109.62:42097
I20260811 19:00:23.937739 20272 heartbeater.cc:464] Registering TS with master...
I20260811 19:00:23.938366 20272 heartbeater.cc:511] Master 127.13.109.62:42097 requested a full tablet report, sending...
I20260811 19:00:23.939586 19774 ts_manager.cc:194] Registered new tserver with Master: 5944cc29b3004a0c968623e7aef7611b (127.13.109.3:34847)
I20260811 19:00:23.940212 19774 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.109.3@KRBTEST.COM'} at 127.13.109.3:41081
I20260811 19:00:23.944473 13748 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/krb5kdc/test-admin.keytab.
Aug 11 19:00:23 dist-test-slave-0kls krb5kdc[19728](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474823, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:00:23.963856 13748 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260811 19:00:23.966951 20279 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:00:23.965378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34622 (local address 127.13.109.1:37597)
0811 19:00:23.965682 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:23.965690 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:23.965728 (+ 38us) server_negotiation.cc:407] Connection header received
0811 19:00:23.965838 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:23.965846 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:23.966078 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:23.966224 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:23.966736 (+ 512us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34622: BlockingRecv error: recv got EOF from 127.0.0.1:34622 (error 108)
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":138,"threads_started":1}
W20260811 19:00:23.967247 13748 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/krb5kdc/krb5cc)
Aug 11 19:00:23 dist-test-slave-0kls krb5kdc[19728](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474823, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:00:23.968421 13748 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 19:00:23 dist-test-slave-0kls krb5kdc[19728](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474823, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:00:23.969507 13748 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 19:00:23 dist-test-slave-0kls krb5kdc[19728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474823, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.109.1@KRBTEST.COM
I20260811 19:00:23.980115 20279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:23.970643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34626 (local address 127.13.109.1:37597)
0811 19:00:23.970814 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:23.970817 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:23.970880 (+ 63us) server_negotiation.cc:407] Connection header received
0811 19:00:23.970987 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:23.970990 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:23.971039 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:23.971108 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:23.971630 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:23.972454 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:23.973291 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:23.973679 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:23.973763 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:23.976254 (+ 2491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:23.976267 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:23.976284 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:23.976314 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:23.978128 (+ 1814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:23.978574 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:23.978579 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:23.978580 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:23.978662 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:23.978941 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:23.978944 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:23.978945 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:23.979260 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:23.979474 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:23.979667 (+ 193us) server_negotiation.cc:299] Negotiation successful
0811 19:00:23.979905 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
W20260811 19:00:23.984792 20279 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:00:23.983499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34638 (local address 127.13.109.1:37597)
0811 19:00:23.983680 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:23.983684 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:23.983802 (+ 118us) server_negotiation.cc:407] Connection header received
0811 19:00:23.983982 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:23.983985 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:23.984028 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:23.984107 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:23.984633 (+ 526us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34638: BlockingRecv error: recv got EOF from 127.0.0.1:34638 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
W20260811 19:00:23.984918 13748 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/krb5kdc/krb5cc' of type FILE
Aug 11 19:00:23 dist-test-slave-0kls krb5kdc[19728](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474823, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:00:23.986123 13748 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 19:00:23 dist-test-slave-0kls krb5kdc[19728](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474823, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:00:23.987077 13748 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 19:00:23 dist-test-slave-0kls krb5kdc[19728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474823, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.109.1@KRBTEST.COM
I20260811 19:00:23.996364 20279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:23.988036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34644 (local address 127.13.109.1:37597)
0811 19:00:23.988186 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:23.988190 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:23.988357 (+ 167us) server_negotiation.cc:407] Connection header received
0811 19:00:23.988463 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:23.988466 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:23.988512 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:23.988582 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:23.989098 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:23.989634 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:23.990492 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:23.990737 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:23.990820 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:23.993145 (+ 2325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:23.993148 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:23.993151 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:23.993178 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:23.994699 (+ 1521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:23.995133 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:23.995136 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:23.995138 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:23.995223 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:23.995498 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:23.995501 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:23.995502 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:23.995701 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:23.995815 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:23.996023 (+ 208us) server_negotiation.cc:299] Negotiation successful
0811 19:00:23.996163 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
W20260811 19:00:23.999513 20279 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:00:23.998427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34652 (local address 127.13.109.1:37597)
0811 19:00:23.998568 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:23.998573 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:23.998720 (+ 147us) server_negotiation.cc:407] Connection header received
0811 19:00:23.998802 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:23.998804 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:23.998847 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:23.998921 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:23.999388 (+ 467us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34652: BlockingRecv error: recv got EOF from 127.0.0.1:34652 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260811 19:00:23.999555 13748 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474792496492-13748-0/krb5kdc/krb5cc' of type FILE
Aug 11 19:00:23 dist-test-slave-0kls krb5kdc[19728](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474823, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:00:24.000687 13748 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 19:00:23 dist-test-slave-0kls krb5kdc[19728](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474823, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:00:24.001622 13748 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 19:00:24 dist-test-slave-0kls krb5kdc[19728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474823, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.109.1@KRBTEST.COM
I20260811 19:00:24.014940 20279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:24.005576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34656 (local address 127.13.109.1:37597)
0811 19:00:24.005736 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:24.005743 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:24.005921 (+ 178us) server_negotiation.cc:407] Connection header received
0811 19:00:24.006113 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:24.006123 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:24.006204 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:24.006298 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:24.007175 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:24.007769 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:24.008744 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:24.008979 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:24.009103 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:24.011690 (+ 2587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:24.011694 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:24.011697 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:24.011734 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:24.013148 (+ 1414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:24.013599 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:24.013603 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:24.013605 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:24.013695 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:24.014016 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:24.014020 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:24.014022 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:24.014240 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:24.014406 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:24.014646 (+ 240us) server_negotiation.cc:299] Negotiation successful
0811 19:00:24.014771 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 19:00:24.016249 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 19845
I20260811 19:00:24.032013 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 19990
I20260811 19:00:24.042145 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 20134
I20260811 19:00:24.049876 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 19741
2026-08-11T19:00:24Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2381 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:00:24 dist-test-slave-0kls krb5kdc[20293](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:00:24 dist-test-slave-0kls krb5kdc[20293](info): set up 2 sockets
Aug 11 19:00:24 dist-test-slave-0kls krb5kdc[20293](info): commencing operation
krb5kdc: starting...
W20260811 19:00:25.151434 13748 mini_kdc.cc:121] Time spent starting KDC: real 1.076s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/krb5kdc/test-user.keytab.
Aug 11 19:00:25 dist-test-slave-0kls krb5kdc[20293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474825, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:00:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:00:25Z Disabled control of system clock
WARNING: no policy specified for oryx/127.13.109.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.109.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.109.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/krb5kdc/oryx_127.13.109.62.keytab.
Entry for principal oryx/127.13.109.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/krb5kdc/oryx_127.13.109.62.keytab.
I20260811 19:00:25.286270 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.62:42815
--webserver_interface=127.13.109.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/krb5kdc/oryx_127.13.109.62.keytab
--principal=oryx/127.13.109.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:36645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.109.62:42815
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:00:25.404844 20308 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:00:25.405190 20308 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:00:25.405268 20308 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:00:25.409431 20308 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:00:25.409547 20308 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:00:25.409585 20308 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:00:25.409624 20308 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:00:25.409659 20308 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:00:25.415138 20308 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:36645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.109.62:42815
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/krb5kdc/oryx_127.13.109.62.keytab
--principal=oryx/127.13.109.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.62:42815
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.109.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20308
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:00:25.416317 20308 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:00:25.417382 20308 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:00:25.426347 20317 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:25.426416 20314 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:25.426412 20308 server_base.cc:1034] running on GCE node
W20260811 19:00:25.426355 20315 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:25.427212 20308 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:00:25.428411 20308 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:00:25.429617 20308 hybrid_clock.cc:648] HybridClock initialized: now 1786474825429603 us; error 30 us; skew 500 ppm
Aug 11 19:00:25 dist-test-slave-0kls krb5kdc[20293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474825, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.109.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:00:25.432685 20308 init.cc:377] Logged in from keytab as oryx/127.13.109.62@KRBTEST.COM (short username oryx)
I20260811 19:00:25.433682 20308 webserver.cc:460] Webserver started at http://127.13.109.62:34529/ using document root <none> and password file <none>
I20260811 19:00:25.434302 20308 fs_manager.cc:359] Metadata directory not provided
I20260811 19:00:25.434399 20308 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:00:25.434669 20308 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:00:25.436854 20308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/master-0/data/instance:
uuid: "449f60a2c460491d97c80e5cd420ef6e"
format_stamp: "Formatted at 2026-08-11 19:00:25 on dist-test-slave-0kls"
server_key: "96861daa7daf83282d60b41f3c919fe3"
server_key_iv: "299bc62f0597013adb06d2ab844e707c"
server_key_version: "encryptionkey@0"
I20260811 19:00:25.437467 20308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/master-0/wal/instance:
uuid: "449f60a2c460491d97c80e5cd420ef6e"
format_stamp: "Formatted at 2026-08-11 19:00:25 on dist-test-slave-0kls"
server_key: "96861daa7daf83282d60b41f3c919fe3"
server_key_iv: "299bc62f0597013adb06d2ab844e707c"
server_key_version: "encryptionkey@0"
I20260811 19:00:25.441493 20308 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260811 19:00:25.444206 20325 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:25.445454 20308 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 19:00:25.445621 20308 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/master-0/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/master-0/wal
uuid: "449f60a2c460491d97c80e5cd420ef6e"
format_stamp: "Formatted at 2026-08-11 19:00:25 on dist-test-slave-0kls"
server_key: "96861daa7daf83282d60b41f3c919fe3"
server_key_iv: "299bc62f0597013adb06d2ab844e707c"
server_key_version: "encryptionkey@0"
I20260811 19:00:25.445750 20308 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:00:25.466974 20308 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:00:25.470440 20308 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:00:25.470690 20308 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:00:25.478109 20308 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.62:42815
I20260811 19:00:25.478138 20381 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.62:42815 every 8 connection(s)
I20260811 19:00:25.479122 20308 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
I20260811 19:00:25.482029 20382 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:00:25.484910 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 20308
I20260811 19:00:25.485069 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/master-0/wal/instance
I20260811 19:00:25.485395 13748 external_mini_cluster.cc:1428] Setting key bcac37805785a902074a9e3516bbb5c9
I20260811 19:00:25.488514 20382 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 449f60a2c460491d97c80e5cd420ef6e: Bootstrap starting.
I20260811 19:00:25.490878 20382 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 449f60a2c460491d97c80e5cd420ef6e: Neither blocks nor log segments found. Creating new log.
I20260811 19:00:25.491649 20382 log.cc:824] T 00000000000000000000000000000000 P 449f60a2c460491d97c80e5cd420ef6e: Log is configured to *not* fsync() on all Append() calls
I20260811 19:00:25.493597 20382 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 449f60a2c460491d97c80e5cd420ef6e: No bootstrap required, opened a new log
Aug 11 19:00:25 dist-test-slave-0kls krb5kdc[20293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474825, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM
I20260811 19:00:25.497725 20382 raft_consensus.cc:348] T 00000000000000000000000000000000 P 449f60a2c460491d97c80e5cd420ef6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "449f60a2c460491d97c80e5cd420ef6e" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 42815 } }
I20260811 19:00:25.498023 20382 raft_consensus.cc:374] T 00000000000000000000000000000000 P 449f60a2c460491d97c80e5cd420ef6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:00:25.498112 20382 raft_consensus.cc:729] T 00000000000000000000000000000000 P 449f60a2c460491d97c80e5cd420ef6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 449f60a2c460491d97c80e5cd420ef6e, State: Initialized, Role: FOLLOWER
I20260811 19:00:25.498618 20382 consensus_queue.cc:260] T 00000000000000000000000000000000 P 449f60a2c460491d97c80e5cd420ef6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "449f60a2c460491d97c80e5cd420ef6e" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 42815 } }
I20260811 19:00:25.498806 20382 raft_consensus.cc:388] T 00000000000000000000000000000000 P 449f60a2c460491d97c80e5cd420ef6e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:00:25.498926 20382 raft_consensus.cc:482] T 00000000000000000000000000000000 P 449f60a2c460491d97c80e5cd420ef6e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:00:25.499063 20382 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 449f60a2c460491d97c80e5cd420ef6e [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:00:25.499984 20386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:25.486998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50158 (local address 127.13.109.62:42815)
0811 19:00:25.487448 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:25.487464 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:25.487500 (+ 36us) server_negotiation.cc:407] Connection header received
0811 19:00:25.488140 (+ 640us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:25.488158 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:25.488488 (+ 330us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:25.488864 (+ 376us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:25.489687 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:25.490762 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:25.491685 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:25.492212 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:25.492390 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:25.495215 (+ 2825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:25.495220 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:25.495233 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:25.495260 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:25.496940 (+ 1680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:25.497444 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:25.497448 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:25.497452 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:25.497561 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:25.497869 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:25.497871 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:25.497872 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:25.498379 (+ 507us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:25.498533 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:25.498856 (+ 323us) server_negotiation.cc:299] Negotiation successful
0811 19:00:25.499223 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":136,"threads_started":1}
I20260811 19:00:25.500030 20382 raft_consensus.cc:504] T 00000000000000000000000000000000 P 449f60a2c460491d97c80e5cd420ef6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "449f60a2c460491d97c80e5cd420ef6e" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 42815 } }
I20260811 19:00:25.500414 20382 leader_election.cc:302] T 00000000000000000000000000000000 P 449f60a2c460491d97c80e5cd420ef6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 449f60a2c460491d97c80e5cd420ef6e; no voters:
I20260811 19:00:25.500733 20382 leader_election.cc:288] T 00000000000000000000000000000000 P 449f60a2c460491d97c80e5cd420ef6e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:00:25.500845 20388 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 449f60a2c460491d97c80e5cd420ef6e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:00:25.501149 20388 raft_consensus.cc:686] T 00000000000000000000000000000000 P 449f60a2c460491d97c80e5cd420ef6e [term 1 LEADER]: Becoming Leader. State: Replica: 449f60a2c460491d97c80e5cd420ef6e, State: Running, Role: LEADER
I20260811 19:00:25.501497 20388 consensus_queue.cc:237] T 00000000000000000000000000000000 P 449f60a2c460491d97c80e5cd420ef6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "449f60a2c460491d97c80e5cd420ef6e" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 42815 } }
I20260811 19:00:25.502024 20382 sys_catalog.cc:564] T 00000000000000000000000000000000 P 449f60a2c460491d97c80e5cd420ef6e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:00:25.503472 20390 sys_catalog.cc:455] T 00000000000000000000000000000000 P 449f60a2c460491d97c80e5cd420ef6e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 449f60a2c460491d97c80e5cd420ef6e. Latest consensus state: current_term: 1 leader_uuid: "449f60a2c460491d97c80e5cd420ef6e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "449f60a2c460491d97c80e5cd420ef6e" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 42815 } } }
I20260811 19:00:25.503489 20389 sys_catalog.cc:455] T 00000000000000000000000000000000 P 449f60a2c460491d97c80e5cd420ef6e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "449f60a2c460491d97c80e5cd420ef6e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "449f60a2c460491d97c80e5cd420ef6e" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 42815 } } }
I20260811 19:00:25.503728 20390 sys_catalog.cc:458] T 00000000000000000000000000000000 P 449f60a2c460491d97c80e5cd420ef6e [sys.catalog]: This master's current role is: LEADER
I20260811 19:00:25.503769 20389 sys_catalog.cc:458] T 00000000000000000000000000000000 P 449f60a2c460491d97c80e5cd420ef6e [sys.catalog]: This master's current role is: LEADER
W20260811 19:00:25.507170 20404 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 449f60a2c460491d97c80e5cd420ef6e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 19:00:25.507238 20404 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 19:00:25.507324 20400 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:00:25.508024 20400 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:00:25.513736 20400 catalog_manager.cc:1347] Generated new cluster ID: 2362e23819bd411a8fb9f457d6f232c5
I20260811 19:00:25.513823 20400 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:00:25.545451 20400 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:00:25.546535 20400 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:00:25.553828 20400 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 449f60a2c460491d97c80e5cd420ef6e: Generated new TSK 0
I20260811 19:00:25.554559 20400 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.13.109.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.109.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.109.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/krb5kdc/oryx_127.13.109.1.keytab.
Entry for principal oryx/127.13.109.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/krb5kdc/oryx_127.13.109.1.keytab.
I20260811 19:00:25.596510 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.1:0
--local_ip_for_outbound_sockets=127.13.109.1
--webserver_interface=127.13.109.1
--webserver_port=0
--tserver_master_addrs=127.13.109.62:42815
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/krb5kdc/oryx_127.13.109.1.keytab
--principal=oryx/127.13.109.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:36645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:00:25.713717 20410 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:00:25.714013 20410 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:00:25.714057 20410 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:00:25.718257 20410 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:00:25.718327 20410 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:00:25.718434 20410 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.1
I20260811 19:00:25.724263 20410 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:36645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/krb5kdc/oryx_127.13.109.1.keytab
--principal=oryx/127.13.109.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.109.1
--webserver_port=0
--tserver_master_addrs=127.13.109.62:42815
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20410
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.1
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:00:25.725410 20410 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:00:25.726590 20410 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:00:25.736822 20416 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:25.737022 20419 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:25.736824 20415 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:25.737915 20410 server_base.cc:1034] running on GCE node
I20260811 19:00:25.738404 20410 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:00:25.739096 20410 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:00:25.740278 20410 hybrid_clock.cc:648] HybridClock initialized: now 1786474825740260 us; error 32 us; skew 500 ppm
Aug 11 19:00:25 dist-test-slave-0kls krb5kdc[20293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474825, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.109.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:00:25.743387 20410 init.cc:377] Logged in from keytab as oryx/127.13.109.1@KRBTEST.COM (short username oryx)
I20260811 19:00:25.744493 20410 webserver.cc:460] Webserver started at http://127.13.109.1:36063/ using document root <none> and password file <none>
I20260811 19:00:25.745077 20410 fs_manager.cc:359] Metadata directory not provided
I20260811 19:00:25.745144 20410 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:00:25.745414 20410 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:00:25.747756 20410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/ts-0/data/instance:
uuid: "2c9427fe9258493f9465751fef6bb43a"
format_stamp: "Formatted at 2026-08-11 19:00:25 on dist-test-slave-0kls"
server_key: "6ebeeb2e88231fd5c41e982cc1f7a1f1"
server_key_iv: "f1be4741d75737dd30357c028c0172aa"
server_key_version: "encryptionkey@0"
I20260811 19:00:25.748399 20410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/ts-0/wal/instance:
uuid: "2c9427fe9258493f9465751fef6bb43a"
format_stamp: "Formatted at 2026-08-11 19:00:25 on dist-test-slave-0kls"
server_key: "6ebeeb2e88231fd5c41e982cc1f7a1f1"
server_key_iv: "f1be4741d75737dd30357c028c0172aa"
server_key_version: "encryptionkey@0"
I20260811 19:00:25.752772 20410 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 19:00:25.755503 20426 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:25.756697 20410 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 19:00:25.756848 20410 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/ts-0/wal
uuid: "2c9427fe9258493f9465751fef6bb43a"
format_stamp: "Formatted at 2026-08-11 19:00:25 on dist-test-slave-0kls"
server_key: "6ebeeb2e88231fd5c41e982cc1f7a1f1"
server_key_iv: "f1be4741d75737dd30357c028c0172aa"
server_key_version: "encryptionkey@0"
I20260811 19:00:25.756966 20410 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:00:25.771335 20410 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:00:25.774880 20410 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:00:25.775089 20410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:00:25.775740 20410 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:00:25.776664 20410 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:00:25.776741 20410 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:25.776810 20410 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:00:25.776863 20410 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:25.786201 20410 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.1:38457
I20260811 19:00:25.786234 20548 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.1:38457 every 8 connection(s)
I20260811 19:00:25.787186 20410 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
I20260811 19:00:25.794345 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 20410
I20260811 19:00:25.794478 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/ts-0/wal/instance
I20260811 19:00:25.794812 13748 external_mini_cluster.cc:1428] Setting key 4494c104a20935ffee34b206ebdd8bdb
Aug 11 19:00:25 dist-test-slave-0kls krb5kdc[20293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474825, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.109.1@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM
I20260811 19:00:25.802389 20386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:25.789166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.1:46549 (local address 127.13.109.62:42815)
0811 19:00:25.789321 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:25.789325 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:25.790231 (+ 906us) server_negotiation.cc:407] Connection header received
0811 19:00:25.791549 (+ 1318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:25.791552 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:25.791604 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:25.791682 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:25.792835 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:25.793413 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:25.794897 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:25.795170 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:25.795289 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:25.798268 (+ 2979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:25.798271 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:25.798273 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:25.798297 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:25.800004 (+ 1707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:25.800541 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:25.800547 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:25.800552 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:25.800651 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:25.800985 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:25.800991 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:25.800995 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:25.801271 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:25.801377 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:25.801992 (+ 615us) server_negotiation.cc:299] Negotiation successful
0811 19:00:25.802168 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 19:00:25.803323 20551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:25.789561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:42815 (local address 127.13.109.1:46549)
0811 19:00:25.790074 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:25.790118 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:25.791353 (+ 1235us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:25.791854 (+ 501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:25.791860 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:25.792226 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:25.792706 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:25.792719 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:25.793540 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:25.793542 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:25.794759 (+ 1217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:25.794770 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:25.795335 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:25.795339 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:25.795517 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:25.796181 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:25.796216 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:25.798089 (+ 1873us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:00:25.800136 (+ 2047us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:25.800141 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:25.800155 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:25.800420 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:25.800746 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:25.800748 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:25.800749 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:25.800882 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:25.801374 (+ 492us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:00:25.801379 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:00:25.801623 (+ 244us) client_negotiation.cc:769] Sending connection context
0811 19:00:25.801892 (+ 269us) client_negotiation.cc:240] Negotiation successful
0811 19:00:25.802235 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":171,"threads_started":1}
I20260811 19:00:25.804705 20549 heartbeater.cc:347] Connected to a master server at 127.13.109.62:42815
I20260811 19:00:25.804955 20549 heartbeater.cc:464] Registering TS with master...
I20260811 19:00:25.805485 20549 heartbeater.cc:511] Master 127.13.109.62:42815 requested a full tablet report, sending...
WARNING: no policy specified for oryx/127.13.109.2@KRBTEST.COM; defaulting to no policy
I20260811 19:00:25.807181 20343 ts_manager.cc:194] Registered new tserver with Master: 2c9427fe9258493f9465751fef6bb43a (127.13.109.1:38457)
I20260811 19:00:25.808395 20343 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.109.1@KRBTEST.COM'} at 127.13.109.1:46549
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.109.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.109.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/krb5kdc/oryx_127.13.109.2.keytab.
Entry for principal oryx/127.13.109.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/krb5kdc/oryx_127.13.109.2.keytab.
I20260811 19:00:25.826995 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.2:0
--local_ip_for_outbound_sockets=127.13.109.2
--webserver_interface=127.13.109.2
--webserver_port=0
--tserver_master_addrs=127.13.109.62:42815
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/krb5kdc/oryx_127.13.109.2.keytab
--principal=oryx/127.13.109.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:36645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:00:25.947306 20554 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:00:25.947628 20554 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:00:25.947674 20554 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:00:25.951978 20554 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:00:25.952093 20554 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:00:25.952160 20554 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.2
I20260811 19:00:25.957669 20554 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:36645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/krb5kdc/oryx_127.13.109.2.keytab
--principal=oryx/127.13.109.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.109.2
--webserver_port=0
--tserver_master_addrs=127.13.109.62:42815
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20554
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.2
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:00:25.958873 20554 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:00:25.959924 20554 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:00:25.970036 20560 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:25.970057 20559 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:25.970090 20562 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:25.970253 20554 server_base.cc:1034] running on GCE node
I20260811 19:00:25.970860 20554 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:00:25.971580 20554 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:00:25.972754 20554 hybrid_clock.cc:648] HybridClock initialized: now 1786474825972736 us; error 28 us; skew 500 ppm
Aug 11 19:00:25 dist-test-slave-0kls krb5kdc[20293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474825, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.109.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:00:25.975875 20554 init.cc:377] Logged in from keytab as oryx/127.13.109.2@KRBTEST.COM (short username oryx)
I20260811 19:00:25.977010 20554 webserver.cc:460] Webserver started at http://127.13.109.2:45991/ using document root <none> and password file <none>
I20260811 19:00:25.977646 20554 fs_manager.cc:359] Metadata directory not provided
I20260811 19:00:25.977744 20554 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:00:25.978020 20554 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:00:25.980293 20554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/ts-1/data/instance:
uuid: "adf5bc4ddc704fec824189012815251a"
format_stamp: "Formatted at 2026-08-11 19:00:25 on dist-test-slave-0kls"
server_key: "f45197dda9c23035b014f6bfd149109d"
server_key_iv: "4d5cab164c4d78df9448cd585c3d5b5c"
server_key_version: "encryptionkey@0"
I20260811 19:00:25.980916 20554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/ts-1/wal/instance:
uuid: "adf5bc4ddc704fec824189012815251a"
format_stamp: "Formatted at 2026-08-11 19:00:25 on dist-test-slave-0kls"
server_key: "f45197dda9c23035b014f6bfd149109d"
server_key_iv: "4d5cab164c4d78df9448cd585c3d5b5c"
server_key_version: "encryptionkey@0"
I20260811 19:00:25.985126 20554 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260811 19:00:25.987983 20569 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:25.989424 20554 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 19:00:25.989620 20554 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/ts-1/wal
uuid: "adf5bc4ddc704fec824189012815251a"
format_stamp: "Formatted at 2026-08-11 19:00:25 on dist-test-slave-0kls"
server_key: "f45197dda9c23035b014f6bfd149109d"
server_key_iv: "4d5cab164c4d78df9448cd585c3d5b5c"
server_key_version: "encryptionkey@0"
I20260811 19:00:25.989761 20554 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:00:26.003656 20554 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:00:26.007480 20554 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:00:26.007706 20554 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:00:26.008729 20554 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:00:26.010028 20554 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:00:26.010104 20554 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:26.010166 20554 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:00:26.010196 20554 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:26.020290 20554 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.2:32839
I20260811 19:00:26.020314 20691 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.2:32839 every 8 connection(s)
I20260811 19:00:26.021328 20554 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
I20260811 19:00:26.024917 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 20554
I20260811 19:00:26.025022 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/ts-1/wal/instance
I20260811 19:00:26.025321 13748 external_mini_cluster.cc:1428] Setting key de7bbdf783e81a1f9a3edc95fb633ab7
Aug 11 19:00:26 dist-test-slave-0kls krb5kdc[20293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474825, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.109.2@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM
WARNING: no policy specified for oryx/127.13.109.3@KRBTEST.COM; defaulting to no policy
I20260811 19:00:26.037138 20386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:26.023460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.2:46675 (local address 127.13.109.62:42815)
0811 19:00:26.023610 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:26.023614 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:26.024539 (+ 925us) server_negotiation.cc:407] Connection header received
0811 19:00:26.026663 (+ 2124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:26.026665 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:26.026701 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:26.026803 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:26.028129 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:26.028669 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:26.029916 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:26.030157 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:26.030280 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:26.033201 (+ 2921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:26.033205 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:26.033208 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:26.033244 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:26.034813 (+ 1569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:26.035370 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:26.035374 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:26.035376 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:26.035464 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:26.035854 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:26.035859 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:26.035861 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:26.036078 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:26.036189 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:26.036807 (+ 618us) server_negotiation.cc:299] Negotiation successful
0811 19:00:26.036949 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 19:00:26.038240 20694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:26.023873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:42815 (local address 127.13.109.2:46675)
0811 19:00:26.024366 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:26.024415 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:26.025948 (+ 1533us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:26.027017 (+ 1069us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:26.027026 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:26.027431 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:26.027972 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:26.027987 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:26.028817 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:26.028821 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:26.029728 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:26.029738 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:26.030332 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:26.030336 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:26.030520 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:26.031182 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:26.031200 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:26.032992 (+ 1792us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:00:26.034947 (+ 1955us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:26.034954 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:26.034974 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:26.035258 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:26.035571 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:26.035574 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:26.035576 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:26.035753 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:26.036203 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:00:26.036212 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:00:26.036553 (+ 341us) client_negotiation.cc:769] Sending connection context
0811 19:00:26.036763 (+ 210us) client_negotiation.cc:240] Negotiation successful
0811 19:00:26.037129 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":125,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.109.3@KRBTEST.COM" created.
I20260811 19:00:26.039551 20692 heartbeater.cc:347] Connected to a master server at 127.13.109.62:42815
I20260811 19:00:26.039865 20692 heartbeater.cc:464] Registering TS with master...
I20260811 19:00:26.040584 20692 heartbeater.cc:511] Master 127.13.109.62:42815 requested a full tablet report, sending...
I20260811 19:00:26.041931 20343 ts_manager.cc:194] Registered new tserver with Master: adf5bc4ddc704fec824189012815251a (127.13.109.2:32839)
I20260811 19:00:26.042577 20343 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.109.2@KRBTEST.COM'} at 127.13.109.2:46675
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.109.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/krb5kdc/oryx_127.13.109.3.keytab.
Entry for principal oryx/127.13.109.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/krb5kdc/oryx_127.13.109.3.keytab.
I20260811 19:00:26.057324 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.3:0
--local_ip_for_outbound_sockets=127.13.109.3
--webserver_interface=127.13.109.3
--webserver_port=0
--tserver_master_addrs=127.13.109.62:42815
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/krb5kdc/oryx_127.13.109.3.keytab
--principal=oryx/127.13.109.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:36645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:00:26.176692 20697 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:00:26.177003 20697 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:00:26.177086 20697 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:00:26.181214 20697 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:00:26.181318 20697 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:00:26.181416 20697 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.3
I20260811 19:00:26.186851 20697 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:36645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/krb5kdc/oryx_127.13.109.3.keytab
--principal=oryx/127.13.109.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.109.3
--webserver_port=0
--tserver_master_addrs=127.13.109.62:42815
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20697
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.3
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:00:26.188021 20697 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:00:26.189090 20697 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 19:00:26.199162 20697 server_base.cc:1034] running on GCE node
W20260811 19:00:26.199134 20703 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:26.199201 20705 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:26.199200 20702 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:26.199858 20697 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:00:26.200453 20697 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:00:26.201617 20697 hybrid_clock.cc:648] HybridClock initialized: now 1786474826201605 us; error 24 us; skew 500 ppm
Aug 11 19:00:26 dist-test-slave-0kls krb5kdc[20293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474826, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.109.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:00:26.204648 20697 init.cc:377] Logged in from keytab as oryx/127.13.109.3@KRBTEST.COM (short username oryx)
I20260811 19:00:26.205585 20697 webserver.cc:460] Webserver started at http://127.13.109.3:36667/ using document root <none> and password file <none>
I20260811 19:00:26.206200 20697 fs_manager.cc:359] Metadata directory not provided
I20260811 19:00:26.206264 20697 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:00:26.206442 20697 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:00:26.208459 20697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/ts-2/data/instance:
uuid: "20f4f9eaa6844d8b9466235cf97ec8ce"
format_stamp: "Formatted at 2026-08-11 19:00:26 on dist-test-slave-0kls"
server_key: "1d09b557eaa214599a885872d16287fc"
server_key_iv: "9b1bb1153e3e2c6ee87d2b546e6b366c"
server_key_version: "encryptionkey@0"
I20260811 19:00:26.208988 20697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/ts-2/wal/instance:
uuid: "20f4f9eaa6844d8b9466235cf97ec8ce"
format_stamp: "Formatted at 2026-08-11 19:00:26 on dist-test-slave-0kls"
server_key: "1d09b557eaa214599a885872d16287fc"
server_key_iv: "9b1bb1153e3e2c6ee87d2b546e6b366c"
server_key_version: "encryptionkey@0"
I20260811 19:00:26.212713 20697 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260811 19:00:26.215205 20712 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:26.216478 20697 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 19:00:26.216615 20697 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/ts-2/wal
uuid: "20f4f9eaa6844d8b9466235cf97ec8ce"
format_stamp: "Formatted at 2026-08-11 19:00:26 on dist-test-slave-0kls"
server_key: "1d09b557eaa214599a885872d16287fc"
server_key_iv: "9b1bb1153e3e2c6ee87d2b546e6b366c"
server_key_version: "encryptionkey@0"
I20260811 19:00:26.216727 20697 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:00:26.239912 20697 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:00:26.243384 20697 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:00:26.243588 20697 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:00:26.244266 20697 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:00:26.245184 20697 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:00:26.245277 20697 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:26.245399 20697 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:00:26.245428 20697 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:26.254755 20697 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.3:43545
I20260811 19:00:26.254773 20833 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.3:43545 every 8 connection(s)
I20260811 19:00:26.255738 20697 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
I20260811 19:00:26.256091 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 20697
I20260811 19:00:26.256246 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/ts-2/wal/instance
I20260811 19:00:26.256606 13748 external_mini_cluster.cc:1428] Setting key 37239f7dc0883e73b0a27258fb48add6
Aug 11 19:00:26 dist-test-slave-0kls krb5kdc[20293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474826, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.109.3@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM
I20260811 19:00:26.269387 20386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:26.257698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.3:39921 (local address 127.13.109.62:42815)
0811 19:00:26.257894 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:26.257898 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:26.258934 (+ 1036us) server_negotiation.cc:407] Connection header received
0811 19:00:26.260341 (+ 1407us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:26.260344 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:26.260399 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:26.260477 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:26.261740 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:26.262308 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:26.263430 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:26.263665 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:26.263778 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:26.266151 (+ 2373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:26.266154 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:26.266156 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:26.266179 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:26.267387 (+ 1208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:26.267885 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:26.267890 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:26.267892 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:26.267977 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:26.268321 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:26.268326 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:26.268328 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:26.268506 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:26.268619 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:26.269090 (+ 471us) server_negotiation.cc:299] Negotiation successful
0811 19:00:26.269211 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 19:00:26.270350 20836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:26.258275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:42815 (local address 127.13.109.3:39921)
0811 19:00:26.258784 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:26.258844 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:26.260135 (+ 1291us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:26.260657 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:26.260663 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:26.261103 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:26.261588 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:26.261601 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:26.262440 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:26.262444 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:26.263296 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:26.263306 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:26.263820 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:26.263824 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:26.263993 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:26.264508 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:26.264524 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:26.265969 (+ 1445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:00:26.267507 (+ 1538us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:26.267513 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:26.267527 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:26.267781 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:26.268077 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:26.268082 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:26.268087 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:26.268225 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:26.268617 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:00:26.268624 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:00:26.268856 (+ 232us) client_negotiation.cc:769] Sending connection context
0811 19:00:26.269041 (+ 185us) client_negotiation.cc:240] Negotiation successful
0811 19:00:26.269350 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":105,"threads_started":1}
I20260811 19:00:26.271450 20834 heartbeater.cc:347] Connected to a master server at 127.13.109.62:42815
I20260811 19:00:26.271654 20834 heartbeater.cc:464] Registering TS with master...
I20260811 19:00:26.272112 20834 heartbeater.cc:511] Master 127.13.109.62:42815 requested a full tablet report, sending...
I20260811 19:00:26.273098 20343 ts_manager.cc:194] Registered new tserver with Master: 20f4f9eaa6844d8b9466235cf97ec8ce (127.13.109.3:43545)
I20260811 19:00:26.273660 20343 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.109.3@KRBTEST.COM'} at 127.13.109.3:39921
I20260811 19:00:26.283157 13748 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.13.109.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.109.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.109.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/krb5kdc/oryx_127.13.109.61.keytab.
Entry for principal oryx/127.13.109.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/krb5kdc/oryx_127.13.109.61.keytab.
I20260811 19:00:26.313948 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.61:43661
--webserver_interface=127.13.109.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/krb5kdc/oryx_127.13.109.61.keytab
--principal=oryx/127.13.109.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:36645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.109.62:42815,127.13.109.61:43661
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:00:26.433480 20839 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:00:26.433849 20839 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:00:26.433931 20839 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:00:26.438108 20839 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:00:26.438190 20839 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:00:26.438228 20839 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:00:26.438287 20839 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:00:26.438325 20839 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:00:26.443817 20839 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:36645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.13.109.62:42815,127.13.109.61:43661
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/krb5kdc/oryx_127.13.109.61.keytab
--principal=oryx/127.13.109.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.61:43661
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.13.109.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20839
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
W20260811 19:00:26.444106 20839 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.13.109.62:42815,127.13.109.61:43661'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260811 19:00:26.445091 20839 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:00:26.446342 20839 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:00:26.455572 20847 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:26.455530 20845 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:26.455530 20844 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:26.455689 20839 server_base.cc:1034] running on GCE node
I20260811 19:00:26.456512 20839 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:00:26.457764 20839 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:00:26.459023 20839 hybrid_clock.cc:648] HybridClock initialized: now 1786474826458991 us; error 51 us; skew 500 ppm
Aug 11 19:00:26 dist-test-slave-0kls krb5kdc[20293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474826, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.109.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:00:26.462333 20839 init.cc:377] Logged in from keytab as oryx/127.13.109.61@KRBTEST.COM (short username oryx)
I20260811 19:00:26.463316 20839 webserver.cc:460] Webserver started at http://127.13.109.61:37525/ using document root <none> and password file <none>
I20260811 19:00:26.463938 20839 fs_manager.cc:359] Metadata directory not provided
I20260811 19:00:26.464011 20839 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:00:26.464205 20839 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:00:26.466516 20839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/master-1/data/instance:
uuid: "26408052dd7842e1a0a036cce7c682c6"
format_stamp: "Formatted at 2026-08-11 19:00:26 on dist-test-slave-0kls"
server_key: "1b2b5fb8cd91f16184ee5acdc8c50c25"
server_key_iv: "c7c9d524b5abb21a7d7d17334a53434a"
server_key_version: "encryptionkey@0"
I20260811 19:00:26.467133 20839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/master-1/wal/instance:
uuid: "26408052dd7842e1a0a036cce7c682c6"
format_stamp: "Formatted at 2026-08-11 19:00:26 on dist-test-slave-0kls"
server_key: "1b2b5fb8cd91f16184ee5acdc8c50c25"
server_key_iv: "c7c9d524b5abb21a7d7d17334a53434a"
server_key_version: "encryptionkey@0"
I20260811 19:00:26.471372 20839 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 19:00:26.474129 20854 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:26.475464 20839 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 19:00:26.475628 20839 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/master-1/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/master-1/wal
uuid: "26408052dd7842e1a0a036cce7c682c6"
format_stamp: "Formatted at 2026-08-11 19:00:26 on dist-test-slave-0kls"
server_key: "1b2b5fb8cd91f16184ee5acdc8c50c25"
server_key_iv: "c7c9d524b5abb21a7d7d17334a53434a"
server_key_version: "encryptionkey@0"
I20260811 19:00:26.475750 20839 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:00:26.500249 20839 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:00:26.504200 20839 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:00:26.504456 20839 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:00:26.512509 20839 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.61:43661
I20260811 19:00:26.512813 20912 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.61:43661 every 8 connection(s)
I20260811 19:00:26.513705 20839 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/master-1/data/info.pb
I20260811 19:00:26.516767 20913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:00:26.520824 20913 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.109.62" port: 42815 } has no permanent_uuid. Determining permanent_uuid...
I20260811 19:00:26.522114 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 20839
Aug 11 19:00:26 dist-test-slave-0kls krb5kdc[20293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474826, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.109.61@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM
Aug 11 19:00:26 dist-test-slave-0kls krb5kdc[20293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474825, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.13.109.61@KRBTEST.COM
I20260811 19:00:26.535565 20386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:26.523038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50168 (local address 127.13.109.62:42815)
0811 19:00:26.523181 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:26.523186 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:26.524222 (+ 1036us) server_negotiation.cc:407] Connection header received
0811 19:00:26.524930 (+ 708us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:26.524934 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:26.524985 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:26.525080 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:26.526817 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:26.527446 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:26.528698 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:26.528974 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:26.529088 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:26.531603 (+ 2515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:26.531606 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:26.531608 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:26.531635 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:26.533223 (+ 1588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:26.533810 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:26.533817 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:26.533822 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:26.533918 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:26.534331 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:26.534335 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:26.534337 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:26.534583 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:26.534697 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:26.535256 (+ 559us) server_negotiation.cc:299] Negotiation successful
0811 19:00:26.535385 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 19:00:26.536406 20917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:26.523498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:42815 (local address 127.0.0.1:50168)
0811 19:00:26.524056 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:26.524105 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:26.524716 (+ 611us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:26.525934 (+ 1218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:26.525939 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:26.526423 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:26.526631 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:26.526644 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:26.527584 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:26.527587 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:26.528578 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:26.528587 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:26.529151 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:26.529154 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:26.529223 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:26.529922 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:26.529939 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:26.531425 (+ 1486us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:00:26.533336 (+ 1911us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:26.533343 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:26.533353 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:26.533671 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:26.534022 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:26.534025 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:26.534027 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:26.534231 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:26.534695 (+ 464us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:00:26.534702 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:00:26.534987 (+ 285us) client_negotiation.cc:769] Sending connection context
0811 19:00:26.535209 (+ 222us) client_negotiation.cc:240] Negotiation successful
0811 19:00:26.535532 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":356,"mutex_wait_us":65,"spinlock_wait_cycles":13056,"thread_start_us":213,"threads_started":1}
I20260811 19:00:26.536406 20916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:26.523497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52144 (local address 127.13.109.61:43661)
0811 19:00:26.523951 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:26.523965 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:26.524000 (+ 35us) server_negotiation.cc:407] Connection header received
0811 19:00:26.524700 (+ 700us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:26.524717 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:26.524946 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:26.525133 (+ 187us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:26.525929 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:26.527197 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:26.528146 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:26.528815 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:26.528924 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:26.532033 (+ 3109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:26.532042 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:26.532057 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:26.532086 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:26.534049 (+ 1963us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:26.534475 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:26.534480 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:26.534482 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:26.534576 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:26.534856 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:26.534860 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:26.534861 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:26.535137 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:26.535357 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:26.535584 (+ 227us) server_negotiation.cc:299] Negotiation successful
0811 19:00:26.536000 (+ 416us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":317,"thread_start_us":154,"threads_started":1}
I20260811 19:00:26.537856 20913 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.109.61" port: 43661 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 19:00:26 dist-test-slave-0kls krb5kdc[20293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474826, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.109.61@KRBTEST.COM for oryx/127.13.109.61@KRBTEST.COM
I20260811 19:00:26.545308 20917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:26.538384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.61:43661 (local address 127.0.0.1:52160)
0811 19:00:26.538502 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:26.538518 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:26.538616 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:26.538923 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:26.538926 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:26.539064 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:26.539145 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:26.539150 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:26.539829 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:26.539831 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:26.540341 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:26.540349 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:26.540757 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:26.540759 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:26.540819 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:26.541238 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:26.541249 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:26.542480 (+ 1231us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:00:26.544102 (+ 1622us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:26.544104 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:26.544105 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:26.544277 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:26.544505 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:26.544507 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:26.544508 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:26.544577 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:26.544887 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:00:26.544889 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:00:26.544962 (+ 73us) client_negotiation.cc:769] Sending connection context
0811 19:00:26.545038 (+ 76us) client_negotiation.cc:240] Negotiation successful
0811 19:00:26.545161 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260811 19:00:26.545336 20916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:26.538433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52160 (local address 127.13.109.61:43661)
0811 19:00:26.538729 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:26.538733 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:26.538746 (+ 13us) server_negotiation.cc:407] Connection header received
0811 19:00:26.538779 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:26.538781 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:26.538822 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:26.538909 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:26.539242 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:26.539727 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:26.540439 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:26.540661 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:26.540740 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:26.542602 (+ 1862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:26.542605 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:26.542607 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:26.542627 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:26.544003 (+ 1376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:26.544359 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:26.544361 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:26.544362 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:26.544429 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:26.544650 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:26.544652 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:26.544653 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:26.544811 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:26.544884 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:26.545096 (+ 212us) server_negotiation.cc:299] Negotiation successful
0811 19:00:26.545214 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":196}
I20260811 19:00:26.548713 20913 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26408052dd7842e1a0a036cce7c682c6: Bootstrap starting.
I20260811 19:00:26.551287 20913 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 26408052dd7842e1a0a036cce7c682c6: Neither blocks nor log segments found. Creating new log.
I20260811 19:00:26.552024 20913 log.cc:824] T 00000000000000000000000000000000 P 26408052dd7842e1a0a036cce7c682c6: Log is configured to *not* fsync() on all Append() calls
I20260811 19:00:26.554153 20913 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26408052dd7842e1a0a036cce7c682c6: No bootstrap required, opened a new log
I20260811 19:00:26.557077 20913 raft_consensus.cc:348] T 00000000000000000000000000000000 P 26408052dd7842e1a0a036cce7c682c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "449f60a2c460491d97c80e5cd420ef6e" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 42815 } } peers { permanent_uuid: "26408052dd7842e1a0a036cce7c682c6" member_type: VOTER last_known_addr { host: "127.13.109.61" port: 43661 } }
I20260811 19:00:26.557271 20913 raft_consensus.cc:374] T 00000000000000000000000000000000 P 26408052dd7842e1a0a036cce7c682c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:00:26.557305 20913 raft_consensus.cc:729] T 00000000000000000000000000000000 P 26408052dd7842e1a0a036cce7c682c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26408052dd7842e1a0a036cce7c682c6, State: Initialized, Role: FOLLOWER
I20260811 19:00:26.557747 20913 consensus_queue.cc:260] T 00000000000000000000000000000000 P 26408052dd7842e1a0a036cce7c682c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "449f60a2c460491d97c80e5cd420ef6e" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 42815 } } peers { permanent_uuid: "26408052dd7842e1a0a036cce7c682c6" member_type: VOTER last_known_addr { host: "127.13.109.61" port: 43661 } }
I20260811 19:00:26.558332 20919 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26408052dd7842e1a0a036cce7c682c6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "449f60a2c460491d97c80e5cd420ef6e" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 42815 } } peers { permanent_uuid: "26408052dd7842e1a0a036cce7c682c6" member_type: VOTER last_known_addr { host: "127.13.109.61" port: 43661 } } }
I20260811 19:00:26.558490 20919 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26408052dd7842e1a0a036cce7c682c6 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 19:00:26.559044 20913 sys_catalog.cc:564] T 00000000000000000000000000000000 P 26408052dd7842e1a0a036cce7c682c6 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 19:00:26.563997 20930 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 26408052dd7842e1a0a036cce7c682c6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 19:00:26.564110 20930 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 19:00:26.566956 20839 master_runner.cc:428] Detected that this master 26408052dd7842e1a0a036cce7c682c6 is joining an existing cluster
I20260811 19:00:26.567055 20839 master_runner.cc:432] Initiating AddMaster RPC to add 127.13.109.61:43661
I20260811 19:00:26.578780 20936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:26.569258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.61:43661 (local address 127.0.0.1:52176)
0811 19:00:26.569755 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:26.569769 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:26.569902 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:26.570288 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:26.570291 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:26.570531 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:26.570654 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:26.570662 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:26.571680 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:26.571684 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:26.572591 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:26.572602 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:26.573086 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:26.573089 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:26.573270 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:26.574323 (+ 1053us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:26.574335 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:26.574842 (+ 507us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:00:26.577011 (+ 2169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:26.577014 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:26.577016 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:26.577233 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:26.577606 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:26.577609 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:26.577612 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:26.577706 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:26.578220 (+ 514us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:00:26.578223 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:00:26.578321 (+ 98us) client_negotiation.cc:769] Sending connection context
0811 19:00:26.578418 (+ 97us) client_negotiation.cc:240] Negotiation successful
0811 19:00:26.578561 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":414,"mutex_wait_us":21,"thread_start_us":117,"threads_started":1}
I20260811 19:00:26.578918 20916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:26.569652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52176 (local address 127.13.109.61:43661)
0811 19:00:26.569779 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:26.569783 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:26.569854 (+ 71us) server_negotiation.cc:407] Connection header received
0811 19:00:26.570049 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:26.570052 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:26.570099 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:26.570173 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:26.570768 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:26.571541 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:26.572725 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:26.572934 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:26.573074 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:26.574986 (+ 1912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:26.574991 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:26.574993 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:26.575020 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:26.576870 (+ 1850us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:26.577339 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:26.577346 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:26.577348 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:26.577490 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:26.577855 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:26.577862 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:26.577863 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:26.578109 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:26.578225 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:26.578517 (+ 292us) server_negotiation.cc:299] Negotiation successful
0811 19:00:26.578649 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":26,"server-negotiator.queue_time_us":63,"spinlock_wait_cycles":4352}
I20260811 19:00:26.579999 20386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:26.569269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50172 (local address 127.13.109.62:42815)
0811 19:00:26.569434 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:26.569439 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:26.569719 (+ 280us) server_negotiation.cc:407] Connection header received
0811 19:00:26.569969 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:26.569976 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:26.570058 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:26.570161 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:26.570696 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:26.571541 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:26.573849 (+ 2308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:26.574097 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:26.574190 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:26.575507 (+ 1317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:26.575511 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:26.575514 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:26.575543 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:26.577323 (+ 1780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:26.577886 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:26.577890 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:26.577892 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:26.577978 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:26.578322 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:26.578326 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:26.578328 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:26.578604 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:26.578723 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:26.579670 (+ 947us) server_negotiation.cc:299] Negotiation successful
0811 19:00:26.579828 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 19:00:26.580035 20937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:26.569199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:42815 (local address 127.0.0.1:50172)
0811 19:00:26.569617 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:26.569632 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:26.569797 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:26.570178 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:26.570181 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:26.570436 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:26.570561 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:26.570568 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:26.572729 (+ 2161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:26.572732 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:26.573667 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:26.573680 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:26.574220 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:26.574224 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:26.574339 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:26.574837 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:26.574851 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:26.575350 (+ 499us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:00:26.577431 (+ 2081us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:26.577434 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:26.577436 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:26.577691 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:26.578088 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:26.578092 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:26.578094 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:26.578191 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:26.578769 (+ 578us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:00:26.578772 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:00:26.579452 (+ 680us) client_negotiation.cc:769] Sending connection context
0811 19:00:26.579595 (+ 143us) client_negotiation.cc:240] Negotiation successful
0811 19:00:26.579818 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":104,"threads_started":1}
Aug 11 19:00:26 dist-test-slave-0kls krb5kdc[20293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474825, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.109.62@KRBTEST.COM for oryx/127.13.109.61@KRBTEST.COM
I20260811 19:00:26.596467 20938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:26.585278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.61:43661 (local address 127.0.0.1:52186)
0811 19:00:26.585654 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:26.585707 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:26.585873 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:26.586252 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:26.586257 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:26.586631 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:26.586750 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:26.586756 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:26.587780 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:26.587783 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:26.588447 (+ 664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:26.588457 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:26.589091 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:26.589096 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:26.589214 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:26.590201 (+ 987us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:26.590216 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:26.592168 (+ 1952us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:00:26.594295 (+ 2127us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:26.594308 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:26.594319 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:26.594723 (+ 404us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:26.595018 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:26.595024 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:26.595029 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:26.595137 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:26.595588 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:00:26.595599 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:00:26.595789 (+ 190us) client_negotiation.cc:769] Sending connection context
0811 19:00:26.595974 (+ 185us) client_negotiation.cc:240] Negotiation successful
0811 19:00:26.596221 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":108,"threads_started":1}
I20260811 19:00:26.597295 20916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:26.585305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52186 (local address 127.13.109.61:43661)
0811 19:00:26.585427 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:26.585432 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:26.585817 (+ 385us) server_negotiation.cc:407] Connection header received
0811 19:00:26.586056 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:26.586059 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:26.586117 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:26.586205 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:26.586866 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:26.587651 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:26.588570 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:26.588914 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:26.589051 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:26.592332 (+ 3281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:26.592335 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:26.592338 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:26.592366 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:26.594170 (+ 1804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:26.594824 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:26.594827 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:26.594829 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:26.594915 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:26.595236 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:26.595239 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:26.595241 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:26.595454 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:26.595548 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:26.596918 (+ 1370us) server_negotiation.cc:299] Negotiation successful
0811 19:00:26.597065 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 19:00:26.599026 20343 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.13.109.61:43661: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "26408052dd7842e1a0a036cce7c682c6"
member_type: NON_VOTER
last_known_addr {
host: "127.13.109.61"
port: 43661
}
attrs {
promote: true
}
}
dest_uuid: "449f60a2c460491d97c80e5cd420ef6e"
cas_config_opid_index: -1
I20260811 19:00:26.599858 20343 consensus_queue.cc:237] T 00000000000000000000000000000000 P 449f60a2c460491d97c80e5cd420ef6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "449f60a2c460491d97c80e5cd420ef6e" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 42815 } } peers { permanent_uuid: "26408052dd7842e1a0a036cce7c682c6" member_type: NON_VOTER last_known_addr { host: "127.13.109.61" port: 43661 } attrs { promote: true } }
WARNING: no policy specified for oryx/127.13.109.60@KRBTEST.COM; defaulting to no policy
I20260811 19:00:26.602844 20884 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 26408052dd7842e1a0a036cce7c682c6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:00:26.602829 20941 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 449f60a2c460491d97c80e5cd420ef6e [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 26408052dd7842e1a0a036cce7c682c6 (127.13.109.61) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "449f60a2c460491d97c80e5cd420ef6e" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 42815 } } peers { permanent_uuid: "26408052dd7842e1a0a036cce7c682c6" member_type: NON_VOTER last_known_addr { host: "127.13.109.61" port: 43661 } attrs { promote: true } } }
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.109.60@KRBTEST.COM" created.
I20260811 19:00:26.604417 20884 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 26408052dd7842e1a0a036cce7c682c6 [term 1 FOLLOWER]: Refusing update from remote peer 449f60a2c460491d97c80e5cd420ef6e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260811 19:00:26.604748 20941 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.13.109.61:43661
I20260811 19:00:26.604991 20941 sys_catalog.cc:455] T 00000000000000000000000000000000 P 449f60a2c460491d97c80e5cd420ef6e [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "449f60a2c460491d97c80e5cd420ef6e" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "449f60a2c460491d97c80e5cd420ef6e" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 42815 } } peers { permanent_uuid: "26408052dd7842e1a0a036cce7c682c6" member_type: NON_VOTER last_known_addr { host: "127.13.109.61" port: 43661 } attrs { promote: true } } }
I20260811 19:00:26.605088 20941 sys_catalog.cc:458] T 00000000000000000000000000000000 P 449f60a2c460491d97c80e5cd420ef6e [sys.catalog]: This master's current role is: LEADER
I20260811 19:00:26.605294 20941 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 449f60a2c460491d97c80e5cd420ef6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "26408052dd7842e1a0a036cce7c682c6" member_type: NON_VOTER last_known_addr { host: "127.13.109.61" port: 43661 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:00:26.605506 20839 master.cc:537] Master@127.13.109.61:43661 shutting down...
W20260811 19:00:26.607723 20326 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260811 19:00:26.608625 20326 consensus_peers.cc:487] T 00000000000000000000000000000000 P 449f60a2c460491d97c80e5cd420ef6e -> Peer 26408052dd7842e1a0a036cce7c682c6 (127.13.109.61:43661): Couldn't send request to peer 26408052dd7842e1a0a036cce7c682c6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260811 19:00:26.610420 20839 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 26408052dd7842e1a0a036cce7c682c6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:00:26.610726 20839 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 26408052dd7842e1a0a036cce7c682c6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:00:26.610857 20839 tablet_replica.cc:330] T 00000000000000000000000000000000 P 26408052dd7842e1a0a036cce7c682c6: stopping tablet replica
I20260811 19:00:26.627267 20839 master.cc:552] Master@127.13.109.61:43661 shutdown complete.
I20260811 19:00:26.627341 20839 master_runner.cc:305] Clearing existing system tablet
I20260811 19:00:26.627615 20839 fs_manager.cc:359] Metadata directory not provided
I20260811 19:00:26.627671 20839 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:00:26.629645 20839 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260811 19:00:26.631067 20948 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:26.631362 20839 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:00:26.631531 20839 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 19:00:26.631587 20839 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/master-1/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/master-1/wal
uuid: "26408052dd7842e1a0a036cce7c682c6"
format_stamp: "Formatted at 2026-08-11 19:00:26 on dist-test-slave-0kls"
server_key: "1b2b5fb8cd91f16184ee5acdc8c50c25"
server_key_iv: "c7c9d524b5abb21a7d7d17334a53434a"
server_key_version: "encryptionkey@0"
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.109.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/krb5kdc/oryx_127.13.109.60.keytab.
Entry for principal oryx/127.13.109.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/krb5kdc/oryx_127.13.109.60.keytab.
I20260811 19:00:26.633288 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.60:34827
--webserver_interface=127.13.109.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/krb5kdc/oryx_127.13.109.60.keytab
--principal=oryx/127.13.109.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:36645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.109.62:42815,127.13.109.61:43661,127.13.109.60:34827
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 19:00:26.634265 20839 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 26408052dd7842e1a0a036cce7c682c6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:00:26.637251 20839 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 26408052dd7842e1a0a036cce7c682c6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 19:00:26.637377 20839 log.cc:1196] T 00000000000000000000000000000000 P 26408052dd7842e1a0a036cce7c682c6: Deleting WAL directory at /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260811 19:00:26.637658 20839 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 26408052dd7842e1a0a036cce7c682c6: Deleting consensus metadata
I20260811 19:00:26.637970 20839 master_runner.cc:315] Copying system tablet from 127.13.109.62:42815
I20260811 19:00:26.639464 20839 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 26408052dd7842e1a0a036cce7c682c6: tablet copy: Beginning tablet copy session from remote peer at address 127.13.109.62:42815
I20260811 19:00:26.648406 20386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:26.640242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50180 (local address 127.13.109.62:42815)
0811 19:00:26.640392 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:26.640398 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:26.640643 (+ 245us) server_negotiation.cc:407] Connection header received
0811 19:00:26.640838 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:26.640841 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:26.640899 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:26.640996 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:26.641502 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:26.642177 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:26.643102 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:26.643353 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:26.643475 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:26.644764 (+ 1289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:26.644771 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:26.644779 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:26.644806 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:26.646498 (+ 1692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:26.646998 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:26.647002 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:26.647004 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:26.647101 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:26.647417 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:26.647422 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:26.647424 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:26.647654 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:26.647779 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:26.648087 (+ 308us) server_negotiation.cc:299] Negotiation successful
0811 19:00:26.648216 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 19:00:26.648422 20955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:26.640184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:42815 (local address 127.0.0.1:50180)
0811 19:00:26.640535 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:26.640549 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:26.640712 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:26.641039 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:26.641043 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:26.641251 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:26.641366 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:26.641372 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:26.642303 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:26.642307 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:26.642951 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:26.642960 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:26.643485 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:26.643489 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:26.643611 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:26.644207 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:26.644221 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:26.644591 (+ 370us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:00:26.646638 (+ 2047us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:26.646641 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:26.646644 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:26.646883 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:26.647208 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:26.647212 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:26.647215 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:26.647311 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:26.647769 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:00:26.647773 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:00:26.647896 (+ 123us) client_negotiation.cc:769] Sending connection context
0811 19:00:26.648019 (+ 123us) client_negotiation.cc:240] Negotiation successful
0811 19:00:26.648204 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":99,"threads_started":1}
I20260811 19:00:26.649637 20365 tablet_copy_service.cc:140] P 449f60a2c460491d97c80e5cd420ef6e: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 26408052dd7842e1a0a036cce7c682c6 ({username='oryx', principal='oryx/127.13.109.61@KRBTEST.COM'} at 127.0.0.1:50180)
I20260811 19:00:26.649896 20365 tablet_copy_service.cc:161] P 449f60a2c460491d97c80e5cd420ef6e: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 26408052dd7842e1a0a036cce7c682c6 at {username='oryx', principal='oryx/127.13.109.61@KRBTEST.COM'} at 127.0.0.1:50180: session id = 26408052dd7842e1a0a036cce7c682c6-00000000000000000000000000000000
I20260811 19:00:26.651912 20365 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 449f60a2c460491d97c80e5cd420ef6e: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 19:00:26.654100 20839 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260811 19:00:26.656467 20839 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 26408052dd7842e1a0a036cce7c682c6: tablet copy: Starting download of 0 data blocks...
I20260811 19:00:26.656733 20839 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 26408052dd7842e1a0a036cce7c682c6: tablet copy: Starting download of 1 WAL segments...
I20260811 19:00:26.658960 20839 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 26408052dd7842e1a0a036cce7c682c6: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 19:00:26.661713 20365 tablet_copy_service.cc:342] P 449f60a2c460491d97c80e5cd420ef6e: Request end of tablet copy session 26408052dd7842e1a0a036cce7c682c6-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.13.109.61@KRBTEST.COM'} at 127.0.0.1:50180
I20260811 19:00:26.661952 20365 tablet_copy_service.cc:433] P 449f60a2c460491d97c80e5cd420ef6e: ending tablet copy session 26408052dd7842e1a0a036cce7c682c6-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 26408052dd7842e1a0a036cce7c682c6
I20260811 19:00:26.670775 20839 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:00:26.672703 20959 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:26.672731 20956 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:26.673063 20839 server_base.cc:1034] running on GCE node
W20260811 19:00:26.673260 20957 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:26.673516 20839 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:00:26.674016 20839 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:00:26.675172 20839 hybrid_clock.cc:648] HybridClock initialized: now 1786474826675183 us; error 51 us; skew 500 ppm
Aug 11 19:00:26 dist-test-slave-0kls krb5kdc[20293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474826, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.109.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:00:26.676898 20839 init.cc:377] Logged in from keytab as oryx/127.13.109.61@KRBTEST.COM (short username oryx)
I20260811 19:00:26.677556 20839 webserver.cc:460] Webserver started at http://127.13.109.61:38805/ using document root <none> and password file <none>
I20260811 19:00:26.677863 20839 fs_manager.cc:359] Metadata directory not provided
I20260811 19:00:26.677942 20839 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:00:26.679307 20839 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 19:00:26.680505 20966 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:26.680786 20839 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 19:00:26.680903 20839 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/master-1/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/master-1/wal
uuid: "26408052dd7842e1a0a036cce7c682c6"
format_stamp: "Formatted at 2026-08-11 19:00:26 on dist-test-slave-0kls"
server_key: "1b2b5fb8cd91f16184ee5acdc8c50c25"
server_key_iv: "c7c9d524b5abb21a7d7d17334a53434a"
server_key_version: "encryptionkey@0"
I20260811 19:00:26.681026 20839 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:00:26.709482 20839 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:00:26.711931 20839 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:00:26.717914 21023 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.61:43661 every 8 connection(s)
I20260811 19:00:26.718194 20839 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.61:43661
I20260811 19:00:26.718834 20839 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/master-1/data/info.pb
I20260811 19:00:26.720891 21024 sys_catalog.cc:263] Verifying existing consensus state
I20260811 19:00:26.722074 21024 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26408052dd7842e1a0a036cce7c682c6: Bootstrap starting.
I20260811 19:00:26.731655 21024 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26408052dd7842e1a0a036cce7c682c6: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 19:00:26.732051 21024 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26408052dd7842e1a0a036cce7c682c6: Bootstrap complete.
I20260811 19:00:26.732678 21024 raft_consensus.cc:348] T 00000000000000000000000000000000 P 26408052dd7842e1a0a036cce7c682c6 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "449f60a2c460491d97c80e5cd420ef6e" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 42815 } } peers { permanent_uuid: "26408052dd7842e1a0a036cce7c682c6" member_type: NON_VOTER last_known_addr { host: "127.13.109.61" port: 43661 } attrs { promote: true } }
I20260811 19:00:26.732836 21024 raft_consensus.cc:729] T 00000000000000000000000000000000 P 26408052dd7842e1a0a036cce7c682c6 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 26408052dd7842e1a0a036cce7c682c6, State: Initialized, Role: LEARNER
I20260811 19:00:26.732985 21024 consensus_queue.cc:260] T 00000000000000000000000000000000 P 26408052dd7842e1a0a036cce7c682c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "449f60a2c460491d97c80e5cd420ef6e" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 42815 } } peers { permanent_uuid: "26408052dd7842e1a0a036cce7c682c6" member_type: NON_VOTER last_known_addr { host: "127.13.109.61" port: 43661 } attrs { promote: true } }
I20260811 19:00:26.733421 21027 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26408052dd7842e1a0a036cce7c682c6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "449f60a2c460491d97c80e5cd420ef6e" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 42815 } } peers { permanent_uuid: "26408052dd7842e1a0a036cce7c682c6" member_type: NON_VOTER last_known_addr { host: "127.13.109.61" port: 43661 } attrs { promote: true } } }
I20260811 19:00:26.733605 21027 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26408052dd7842e1a0a036cce7c682c6 [sys.catalog]: This master's current role is: LEARNER
I20260811 19:00:26.733624 21024 sys_catalog.cc:564] T 00000000000000000000000000000000 P 26408052dd7842e1a0a036cce7c682c6 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:00:26.737483 21038 catalog_manager.cc:1259] Loaded cluster ID: 2362e23819bd411a8fb9f457d6f232c5
I20260811 19:00:26.737613 21038 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 26408052dd7842e1a0a036cce7c682c6: loading cluster ID for follower catalog manager: success
I20260811 19:00:26.740135 21038 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 26408052dd7842e1a0a036cce7c682c6: acquiring CA information for follower catalog manager: success
I20260811 19:00:26.741958 21038 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 26408052dd7842e1a0a036cce7c682c6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260811 19:00:26.795434 20949 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:00:26.795698 20949 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:00:26.795769 20949 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:00:26.799984 20949 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:00:26.800047 20949 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:00:26.800068 20949 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:00:26.800088 20949 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:00:26.800103 20949 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:00:26.805668 20949 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:36645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.13.109.62:42815,127.13.109.61:43661,127.13.109.60:34827
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/krb5kdc/oryx_127.13.109.60.keytab
--principal=oryx/127.13.109.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.60:34827
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.13.109.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20949
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:00:26.806838 20949 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:00:26.807822 20949 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 19:00:26.811476 20549 heartbeater.cc:503] Master 127.13.109.62:42815 was elected leader, sending a full tablet report...
W20260811 19:00:26.816568 21044 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:26.816545 21043 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:26.816546 21046 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:26.816745 20949 server_base.cc:1034] running on GCE node
I20260811 19:00:26.817409 20949 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:00:26.818629 20949 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:00:26.819792 20949 hybrid_clock.cc:648] HybridClock initialized: now 1786474826819774 us; error 27 us; skew 500 ppm
Aug 11 19:00:26 dist-test-slave-0kls krb5kdc[20293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474826, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.109.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:00:26.822629 20949 init.cc:377] Logged in from keytab as oryx/127.13.109.60@KRBTEST.COM (short username oryx)
I20260811 19:00:26.823553 20949 webserver.cc:460] Webserver started at http://127.13.109.60:45083/ using document root <none> and password file <none>
I20260811 19:00:26.824159 20949 fs_manager.cc:359] Metadata directory not provided
I20260811 19:00:26.824221 20949 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:00:26.824481 20949 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:00:26.826650 20949 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/master-2/data/instance:
uuid: "adbbc0043cb14763a95172b87454443a"
format_stamp: "Formatted at 2026-08-11 19:00:26 on dist-test-slave-0kls"
server_key: "5831d0fd79fcd0c89385005d22cbea15"
server_key_iv: "c25aab5965f2462b552c9b3bcdb9a7b3"
server_key_version: "encryptionkey@0"
I20260811 19:00:26.827234 20949 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/master-2/wal/instance:
uuid: "adbbc0043cb14763a95172b87454443a"
format_stamp: "Formatted at 2026-08-11 19:00:26 on dist-test-slave-0kls"
server_key: "5831d0fd79fcd0c89385005d22cbea15"
server_key_iv: "c25aab5965f2462b552c9b3bcdb9a7b3"
server_key_version: "encryptionkey@0"
I20260811 19:00:26.831123 20949 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 19:00:26.833841 21054 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:26.835278 20949 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 19:00:26.835418 20949 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/master-2/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/master-2/wal
uuid: "adbbc0043cb14763a95172b87454443a"
format_stamp: "Formatted at 2026-08-11 19:00:26 on dist-test-slave-0kls"
server_key: "5831d0fd79fcd0c89385005d22cbea15"
server_key_iv: "c25aab5965f2462b552c9b3bcdb9a7b3"
server_key_version: "encryptionkey@0"
I20260811 19:00:26.835536 20949 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:00:26.873508 20949 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:00:26.876889 20949 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:00:26.877130 20949 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:00:26.884795 20949 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.60:34827
I20260811 19:00:26.885104 21111 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.60:34827 every 8 connection(s)
I20260811 19:00:26.885938 20949 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/master-2/data/info.pb
I20260811 19:00:26.888899 21112 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:00:26.892894 21112 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.109.62" port: 42815 } has no permanent_uuid. Determining permanent_uuid...
I20260811 19:00:26.893385 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 20949
Aug 11 19:00:26 dist-test-slave-0kls krb5kdc[20293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474826, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.109.60@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM
Aug 11 19:00:26 dist-test-slave-0kls krb5kdc[20293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474825, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.13.109.60@KRBTEST.COM
I20260811 19:00:26.906877 20386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:26.895329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50184 (local address 127.13.109.62:42815)
0811 19:00:26.895467 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:26.895471 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:26.895760 (+ 289us) server_negotiation.cc:407] Connection header received
0811 19:00:26.896648 (+ 888us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:26.896651 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:26.896701 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:26.896795 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:26.898002 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:26.898523 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:26.899576 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:26.899824 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:26.899942 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:26.902774 (+ 2832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:26.902778 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:26.902780 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:26.902809 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:26.904511 (+ 1702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:26.905150 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:26.905154 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:26.905156 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:26.905250 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:26.905610 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:26.905614 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:26.905616 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:26.905899 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:26.906012 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:26.906558 (+ 546us) server_negotiation.cc:299] Negotiation successful
0811 19:00:26.906694 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 19:00:26.907760 21115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:26.895132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:42815 (local address 127.0.0.1:50184)
0811 19:00:26.895595 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:26.895642 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:26.896406 (+ 764us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:26.897246 (+ 840us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:26.897250 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:26.897578 (+ 328us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:26.897853 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:26.897866 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:26.898672 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:26.898675 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:26.899445 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:26.899454 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:26.900001 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:26.900005 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:26.900126 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:26.900768 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:26.900785 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:26.902610 (+ 1825us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:00:26.904648 (+ 2038us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:26.904657 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:26.904671 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:26.905041 (+ 370us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:26.905348 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:26.905352 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:26.905354 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:26.905505 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:26.906011 (+ 506us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:00:26.906020 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:00:26.906308 (+ 288us) client_negotiation.cc:769] Sending connection context
0811 19:00:26.906502 (+ 194us) client_negotiation.cc:240] Negotiation successful
0811 19:00:26.906862 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":163,"threads_started":1}
I20260811 19:00:26.907763 21116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:26.895086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46522 (local address 127.13.109.60:34827)
0811 19:00:26.895641 (+ 555us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:26.895654 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:26.895684 (+ 30us) server_negotiation.cc:407] Connection header received
0811 19:00:26.896382 (+ 698us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:26.896402 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:26.896662 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:26.896845 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:26.897589 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:26.898597 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:26.899475 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:26.899870 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:26.900052 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:26.903185 (+ 3133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:26.903194 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:26.903212 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:26.903246 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:26.905130 (+ 1884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:26.905595 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:26.905602 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:26.905603 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:26.905689 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:26.906023 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:26.906027 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:26.906028 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:26.906313 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:26.906542 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:26.906726 (+ 184us) server_negotiation.cc:299] Negotiation successful
0811 19:00:26.907060 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":41,"server-negotiator.queue_time_us":458,"spinlock_wait_cycles":10880,"thread_start_us":183,"threads_started":1}
I20260811 19:00:26.909286 21112 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.109.61" port: 43661 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 19:00:26 dist-test-slave-0kls krb5kdc[20293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474826, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.109.60@KRBTEST.COM for oryx/127.13.109.61@KRBTEST.COM
I20260811 19:00:26.918851 21118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:26.910083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52190 (local address 127.13.109.61:43661)
0811 19:00:26.910407 (+ 324us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:26.910412 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:26.910426 (+ 14us) server_negotiation.cc:407] Connection header received
0811 19:00:26.910471 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:26.910474 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:26.910527 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:26.910634 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:26.911084 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:26.911788 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:26.912609 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:26.912853 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:26.912964 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:26.915143 (+ 2179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:26.915147 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:26.915150 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:26.915179 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:26.916726 (+ 1547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:26.917185 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:26.917188 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:26.917190 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:26.917279 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:26.917572 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:26.917577 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:26.917579 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:26.918040 (+ 461us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:26.918267 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:26.918548 (+ 281us) server_negotiation.cc:299] Negotiation successful
0811 19:00:26.918681 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":242,"thread_start_us":117,"threads_started":1}
I20260811 19:00:26.918848 21115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:26.909909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.61:43661 (local address 127.0.0.1:52190)
0811 19:00:26.910080 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:26.910093 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:26.910184 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:26.910661 (+ 477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:26.910664 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:26.910853 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:26.910955 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:26.910961 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:26.911909 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:26.911914 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:26.912471 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:26.912479 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:26.912975 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:26.912979 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:26.913086 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:26.913624 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:26.913636 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:26.914991 (+ 1355us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:00:26.916860 (+ 1869us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:26.916864 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:26.916866 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:26.917070 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:26.917381 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:26.917384 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:26.917386 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:26.917477 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:26.918249 (+ 772us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:00:26.918253 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:00:26.918373 (+ 120us) client_negotiation.cc:769] Sending connection context
0811 19:00:26.918489 (+ 116us) client_negotiation.cc:240] Negotiation successful
0811 19:00:26.918666 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108}
I20260811 19:00:26.919962 21112 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.109.60" port: 34827 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 19:00:26 dist-test-slave-0kls krb5kdc[20293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474826, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.109.60@KRBTEST.COM for oryx/127.13.109.60@KRBTEST.COM
I20260811 19:00:26.928182 21116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:26.920649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46526 (local address 127.13.109.60:34827)
0811 19:00:26.920774 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:26.920777 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:26.920788 (+ 11us) server_negotiation.cc:407] Connection header received
0811 19:00:26.920817 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:26.920819 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:26.920852 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:26.920917 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:26.921462 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:26.922043 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:26.922793 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:26.923034 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:26.923114 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:26.925183 (+ 2069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:26.925186 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:26.925188 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:26.925211 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:26.926735 (+ 1524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:26.927122 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:26.927125 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:26.927126 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:26.927201 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:26.927437 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:26.927439 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:26.927440 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:26.927621 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:26.927698 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:26.927933 (+ 235us) server_negotiation.cc:299] Negotiation successful
0811 19:00:26.928029 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 19:00:26.928182 21115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:26.920546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.60:34827 (local address 127.0.0.1:46526)
0811 19:00:26.920644 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:26.920656 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:26.920730 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:26.921118 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:26.921121 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:26.921272 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:26.921353 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:26.921358 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:26.922180 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:26.922183 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:26.922679 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:26.922686 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:26.923138 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:26.923140 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:26.923199 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:26.923697 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:26.923708 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:26.925036 (+ 1328us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:00:26.926838 (+ 1802us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:26.926840 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:26.926842 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:26.927031 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:26.927281 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:26.927284 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:26.927285 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:26.927360 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:26.927705 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:00:26.927706 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:00:26.927790 (+ 84us) client_negotiation.cc:769] Sending connection context
0811 19:00:26.927874 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 19:00:26.927983 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260811 19:00:26.931937 21112 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P adbbc0043cb14763a95172b87454443a: Bootstrap starting.
I20260811 19:00:26.934564 21112 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P adbbc0043cb14763a95172b87454443a: Neither blocks nor log segments found. Creating new log.
I20260811 19:00:26.935412 21112 log.cc:824] T 00000000000000000000000000000000 P adbbc0043cb14763a95172b87454443a: Log is configured to *not* fsync() on all Append() calls
I20260811 19:00:26.937589 21112 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P adbbc0043cb14763a95172b87454443a: No bootstrap required, opened a new log
I20260811 19:00:26.941011 21112 raft_consensus.cc:348] T 00000000000000000000000000000000 P adbbc0043cb14763a95172b87454443a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "449f60a2c460491d97c80e5cd420ef6e" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 42815 } } peers { permanent_uuid: "26408052dd7842e1a0a036cce7c682c6" member_type: VOTER last_known_addr { host: "127.13.109.61" port: 43661 } } peers { permanent_uuid: "adbbc0043cb14763a95172b87454443a" member_type: VOTER last_known_addr { host: "127.13.109.60" port: 34827 } }
I20260811 19:00:26.941267 21112 raft_consensus.cc:374] T 00000000000000000000000000000000 P adbbc0043cb14763a95172b87454443a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:00:26.941361 21112 raft_consensus.cc:729] T 00000000000000000000000000000000 P adbbc0043cb14763a95172b87454443a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: adbbc0043cb14763a95172b87454443a, State: Initialized, Role: FOLLOWER
I20260811 19:00:26.941892 21112 consensus_queue.cc:260] T 00000000000000000000000000000000 P adbbc0043cb14763a95172b87454443a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "449f60a2c460491d97c80e5cd420ef6e" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 42815 } } peers { permanent_uuid: "26408052dd7842e1a0a036cce7c682c6" member_type: VOTER last_known_addr { host: "127.13.109.61" port: 43661 } } peers { permanent_uuid: "adbbc0043cb14763a95172b87454443a" member_type: VOTER last_known_addr { host: "127.13.109.60" port: 34827 } }
I20260811 19:00:26.942466 21120 sys_catalog.cc:455] T 00000000000000000000000000000000 P adbbc0043cb14763a95172b87454443a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "449f60a2c460491d97c80e5cd420ef6e" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 42815 } } peers { permanent_uuid: "26408052dd7842e1a0a036cce7c682c6" member_type: VOTER last_known_addr { host: "127.13.109.61" port: 43661 } } peers { permanent_uuid: "adbbc0043cb14763a95172b87454443a" member_type: VOTER last_known_addr { host: "127.13.109.60" port: 34827 } } }
I20260811 19:00:26.942593 21120 sys_catalog.cc:458] T 00000000000000000000000000000000 P adbbc0043cb14763a95172b87454443a [sys.catalog]: This master's current role is: FOLLOWER
I20260811 19:00:26.943259 21112 sys_catalog.cc:564] T 00000000000000000000000000000000 P adbbc0043cb14763a95172b87454443a [sys.catalog]: configured and running, proceeding with master startup.
W20260811 19:00:26.948733 21133 catalog_manager.cc:1558] T 00000000000000000000000000000000 P adbbc0043cb14763a95172b87454443a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 19:00:26.948853 21133 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 19:00:26.952631 20949 master_runner.cc:428] Detected that this master adbbc0043cb14763a95172b87454443a is joining an existing cluster
I20260811 19:00:26.952730 20949 master_runner.cc:432] Initiating AddMaster RPC to add 127.13.109.60:34827
I20260811 19:00:26.962021 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 20410
I20260811 19:00:26.966471 21118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:26.956115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52204 (local address 127.13.109.61:43661)
0811 19:00:26.956384 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:26.956387 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:26.956398 (+ 11us) server_negotiation.cc:407] Connection header received
0811 19:00:26.956436 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:26.956438 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:26.956504 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:26.956579 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:26.957251 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:26.957847 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:26.959302 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:26.959672 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:26.959824 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:26.961202 (+ 1378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:26.961206 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:26.961209 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:26.961238 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:26.964488 (+ 3250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:26.965033 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:26.965037 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:26.965039 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:26.965128 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:26.965430 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:26.965433 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:26.965435 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:26.965677 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:26.965749 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:26.966151 (+ 402us) server_negotiation.cc:299] Negotiation successful
0811 19:00:26.966297 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164}
I20260811 19:00:26.967061 21140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:26.955207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.61:43661 (local address 127.0.0.1:52204)
0811 19:00:26.955707 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:26.955724 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:26.955844 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:26.956806 (+ 962us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:26.956808 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:26.956998 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:26.957136 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:26.957141 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:26.957971 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:26.957974 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:26.959158 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:26.959170 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:26.959834 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:26.959840 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:26.959922 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:26.960551 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:26.960565 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:26.961056 (+ 491us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:00:26.964643 (+ 3587us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:26.964646 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:26.964649 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:26.964915 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:26.965241 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:26.965243 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:26.965245 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:26.965333 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:26.965856 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:00:26.965859 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:00:26.965962 (+ 103us) client_negotiation.cc:769] Sending connection context
0811 19:00:26.966708 (+ 746us) client_negotiation.cc:240] Negotiation successful
0811 19:00:26.966875 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":138,"threads_started":1}
I20260811 19:00:26.973032 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.1:38457
--local_ip_for_outbound_sockets=127.13.109.1
--tserver_master_addrs=127.13.109.62:42815
--webserver_port=36063
--webserver_interface=127.13.109.1
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/krb5kdc/oryx_127.13.109.1.keytab
--principal=oryx/127.13.109.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:36645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.13.109.62:42815,127.13.109.61:43661
--tserver_master_addrs=127.13.109.62:42815,127.13.109.61:43661,127.13.109.60:34827 with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 19:00:26.979972 21116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:26.955776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46538 (local address 127.13.109.60:34827)
0811 19:00:26.955895 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:26.955899 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:26.956249 (+ 350us) server_negotiation.cc:407] Connection header received
0811 19:00:26.956349 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:26.956352 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:26.956399 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:26.956487 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:26.957104 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:26.957933 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:26.959529 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:26.959789 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:26.959883 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:26.975269 (+ 15386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:26.975277 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:26.975281 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:26.975310 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:26.977282 (+ 1972us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:26.978362 (+ 1080us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:26.978369 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:26.978373 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:26.978473 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:26.978790 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:26.978793 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:26.978795 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:26.979065 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:26.979152 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:26.979531 (+ 379us) server_negotiation.cc:299] Negotiation successful
0811 19:00:26.979679 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 19:00:26.980919 21142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:26.955755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.60:34827 (local address 127.0.0.1:46538)
0811 19:00:26.956165 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:26.956177 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:26.956270 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:26.956518 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:26.956522 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:26.956811 (+ 289us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:26.956980 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:26.956990 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:26.958027 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:26.958031 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:26.959419 (+ 1388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:26.959427 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:26.959996 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:26.960000 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:26.960149 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:26.974403 (+ 14254us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:26.974426 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:26.975098 (+ 672us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:00:26.977415 (+ 2317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:26.977421 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:26.977426 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:26.978238 (+ 812us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:26.978588 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:26.978594 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:26.978599 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:26.978694 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:26.979224 (+ 530us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:00:26.979228 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:00:26.979337 (+ 109us) client_negotiation.cc:769] Sending connection context
0811 19:00:26.980463 (+ 1126us) client_negotiation.cc:240] Negotiation successful
0811 19:00:26.980654 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":104,"threads_started":1}
I20260811 19:00:26.982267 21141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:26.955215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:42815 (local address 127.0.0.1:50196)
0811 19:00:26.955737 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:26.955754 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:26.955889 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:26.956707 (+ 818us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:26.956711 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:26.974032 (+ 17321us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:26.974208 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:26.974220 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:26.975408 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:26.975412 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:26.976362 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:26.976373 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:26.977189 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:26.977193 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:26.977298 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:26.977753 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:26.977769 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:26.978363 (+ 594us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:00:26.980438 (+ 2075us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:26.980441 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:26.980443 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:26.980782 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:26.981089 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:26.981093 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:26.981095 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:26.981179 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:26.981635 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:00:26.981638 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:00:26.981842 (+ 204us) client_negotiation.cc:769] Sending connection context
0811 19:00:26.981954 (+ 112us) client_negotiation.cc:240] Negotiation successful
0811 19:00:26.982093 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":437,"mutex_wait_us":21,"thread_start_us":137,"threads_started":1}
I20260811 19:00:26.982583 20386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:26.955197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50196 (local address 127.13.109.62:42815)
0811 19:00:26.955370 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:26.955377 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:26.955828 (+ 451us) server_negotiation.cc:407] Connection header received
0811 19:00:26.956019 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:26.956023 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:26.956089 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:26.956171 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:26.974412 (+ 18241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:26.975255 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:26.976602 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:26.976946 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:26.977047 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:26.978543 (+ 1496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:26.978547 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:26.978550 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:26.978579 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:26.980312 (+ 1733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:26.980896 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:26.980899 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:26.980901 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:26.980989 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:26.981276 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:26.981279 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:26.981281 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:26.981486 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:26.981589 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:26.982065 (+ 476us) server_negotiation.cc:299] Negotiation successful
0811 19:00:26.982200 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
Aug 11 19:00:26 dist-test-slave-0kls krb5kdc[20293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474825, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.109.62@KRBTEST.COM for oryx/127.13.109.60@KRBTEST.COM
I20260811 19:00:26.997958 21116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:26.987303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46542 (local address 127.13.109.60:34827)
0811 19:00:26.987796 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:26.987803 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:26.987819 (+ 16us) server_negotiation.cc:407] Connection header received
0811 19:00:26.987908 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:26.987913 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:26.987985 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:26.988099 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:26.988661 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:26.989548 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:26.990764 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:26.991148 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:26.991262 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:26.993964 (+ 2702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:26.993968 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:26.993971 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:26.993998 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:26.995750 (+ 1752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:26.996525 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:26.996531 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:26.996534 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:26.996629 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:26.996948 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:26.996951 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:26.996953 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:26.997198 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:26.997299 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:26.997624 (+ 325us) server_negotiation.cc:299] Negotiation successful
0811 19:00:26.997757 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388}
I20260811 19:00:26.997959 20938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:26.987235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.60:34827 (local address 127.0.0.1:46542)
0811 19:00:26.987471 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:26.987490 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:26.987595 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:26.988149 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:26.988153 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:26.988383 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:26.988513 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:26.988521 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:26.989682 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:26.989686 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:26.990631 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:26.990644 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:26.991286 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:26.991289 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:26.991399 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:26.992003 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:26.992024 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:26.993753 (+ 1729us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:00:26.995891 (+ 2138us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:26.995894 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:26.995897 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:26.996169 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:26.996738 (+ 569us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:26.996741 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:26.996744 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:26.996844 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:26.997304 (+ 460us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:00:26.997307 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:00:26.997424 (+ 117us) client_negotiation.cc:769] Sending connection context
0811 19:00:26.997541 (+ 117us) client_negotiation.cc:240] Negotiation successful
0811 19:00:26.997717 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131}
I20260811 19:00:27.000080 20343 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.13.109.60:34827: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "adbbc0043cb14763a95172b87454443a"
member_type: NON_VOTER
last_known_addr {
host: "127.13.109.60"
port: 34827
}
attrs {
promote: true
}
}
dest_uuid: "449f60a2c460491d97c80e5cd420ef6e"
cas_config_opid_index: 5
I20260811 19:00:27.000777 20343 consensus_queue.cc:237] T 00000000000000000000000000000000 P 449f60a2c460491d97c80e5cd420ef6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "449f60a2c460491d97c80e5cd420ef6e" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 42815 } } peers { permanent_uuid: "26408052dd7842e1a0a036cce7c682c6" member_type: NON_VOTER last_known_addr { host: "127.13.109.61" port: 43661 } attrs { promote: true } } peers { permanent_uuid: "adbbc0043cb14763a95172b87454443a" member_type: NON_VOTER last_known_addr { host: "127.13.109.60" port: 34827 } attrs { promote: true } }
I20260811 19:00:27.002027 20942 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 449f60a2c460491d97c80e5cd420ef6e [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER adbbc0043cb14763a95172b87454443a (127.13.109.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "449f60a2c460491d97c80e5cd420ef6e" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 42815 } } peers { permanent_uuid: "26408052dd7842e1a0a036cce7c682c6" member_type: NON_VOTER last_known_addr { host: "127.13.109.61" port: 43661 } attrs { promote: true } } peers { permanent_uuid: "adbbc0043cb14763a95172b87454443a" member_type: NON_VOTER last_known_addr { host: "127.13.109.60" port: 34827 } attrs { promote: true } } }
I20260811 19:00:27.003695 20942 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.13.109.60:34827
I20260811 19:00:27.003909 20942 sys_catalog.cc:455] T 00000000000000000000000000000000 P 449f60a2c460491d97c80e5cd420ef6e [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "449f60a2c460491d97c80e5cd420ef6e" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "449f60a2c460491d97c80e5cd420ef6e" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 42815 } } peers { permanent_uuid: "26408052dd7842e1a0a036cce7c682c6" member_type: NON_VOTER last_known_addr { host: "127.13.109.61" port: 43661 } attrs { promote: true } } peers { permanent_uuid: "adbbc0043cb14763a95172b87454443a" member_type: NON_VOTER last_known_addr { host: "127.13.109.60" port: 34827 } attrs { promote: true } } }
I20260811 19:00:27.004014 20942 sys_catalog.cc:458] T 00000000000000000000000000000000 P 449f60a2c460491d97c80e5cd420ef6e [sys.catalog]: This master's current role is: LEADER
I20260811 19:00:27.004953 21081 raft_consensus.cc:3037] T 00000000000000000000000000000000 P adbbc0043cb14763a95172b87454443a [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:00:27.006551 21081 raft_consensus.cc:1261] T 00000000000000000000000000000000 P adbbc0043cb14763a95172b87454443a [term 1 FOLLOWER]: Refusing update from remote peer 449f60a2c460491d97c80e5cd420ef6e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260811 19:00:27.007411 20938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:27.002055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.61:43661 (local address 127.0.0.1:52216)
0811 19:00:27.002229 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:27.002244 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:27.002332 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:27.002680 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:27.002683 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:27.002701 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:00:27.002843 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:27.002850 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:27.004182 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:27.004186 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:27.005582 (+ 1396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:27.005595 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:27.006927 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:27.006931 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:27.007038 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:27.007092 (+ 54us) client_negotiation.cc:769] Sending connection context
0811 19:00:27.007176 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 19:00:27.007269 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92}
I20260811 19:00:27.007891 21118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:27.002141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52216 (local address 127.13.109.61:43661)
0811 19:00:27.002269 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:27.002275 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:27.002291 (+ 16us) server_negotiation.cc:407] Connection header received
0811 19:00:27.002469 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:27.002471 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:27.002554 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:27.002655 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:00:27.002969 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:27.003936 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:27.005737 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:27.006470 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:27.006595 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:27.006855 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:27.007678 (+ 823us) server_negotiation.cc:299] Negotiation successful
0811 19:00:27.007769 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 19:00:27.007908 20949 master.cc:537] Master@127.13.109.60:34827 shutting down...
I20260811 19:00:27.007478 20941 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 449f60a2c460491d97c80e5cd420ef6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "adbbc0043cb14763a95172b87454443a" member_type: NON_VOTER last_known_addr { host: "127.13.109.60" port: 34827 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:00:27.009691 20995 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 26408052dd7842e1a0a036cce7c682c6 [term 1 LEARNER]: Refusing update from remote peer 449f60a2c460491d97c80e5cd420ef6e: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
W20260811 19:00:27.010766 20326 consensus_peers.cc:487] T 00000000000000000000000000000000 P 449f60a2c460491d97c80e5cd420ef6e -> Peer adbbc0043cb14763a95172b87454443a (127.13.109.60:34827): Couldn't send request to peer adbbc0043cb14763a95172b87454443a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260811 19:00:27.011087 20941 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 449f60a2c460491d97c80e5cd420ef6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "26408052dd7842e1a0a036cce7c682c6" member_type: NON_VOTER last_known_addr { host: "127.13.109.61" port: 43661 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260811 19:00:27.013337 20995 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 26408052dd7842e1a0a036cce7c682c6 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER adbbc0043cb14763a95172b87454443a (127.13.109.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "449f60a2c460491d97c80e5cd420ef6e" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 42815 } } peers { permanent_uuid: "26408052dd7842e1a0a036cce7c682c6" member_type: NON_VOTER last_known_addr { host: "127.13.109.61" port: 43661 } attrs { promote: true } } peers { permanent_uuid: "adbbc0043cb14763a95172b87454443a" member_type: NON_VOTER last_known_addr { host: "127.13.109.60" port: 34827 } attrs { promote: true } } }
I20260811 19:00:27.015831 21027 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26408052dd7842e1a0a036cce7c682c6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 449f60a2c460491d97c80e5cd420ef6e. Latest consensus state: current_term: 1 leader_uuid: "449f60a2c460491d97c80e5cd420ef6e" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "449f60a2c460491d97c80e5cd420ef6e" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 42815 } } peers { permanent_uuid: "26408052dd7842e1a0a036cce7c682c6" member_type: NON_VOTER last_known_addr { host: "127.13.109.61" port: 43661 } attrs { promote: true } } peers { permanent_uuid: "adbbc0043cb14763a95172b87454443a" member_type: NON_VOTER last_known_addr { host: "127.13.109.60" port: 34827 } attrs { promote: true } } }
I20260811 19:00:27.015969 21027 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26408052dd7842e1a0a036cce7c682c6 [sys.catalog]: This master's current role is: LEARNER
I20260811 19:00:27.016076 21027 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26408052dd7842e1a0a036cce7c682c6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "449f60a2c460491d97c80e5cd420ef6e" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "449f60a2c460491d97c80e5cd420ef6e" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 42815 } } peers { permanent_uuid: "26408052dd7842e1a0a036cce7c682c6" member_type: NON_VOTER last_known_addr { host: "127.13.109.61" port: 43661 } attrs { promote: true } } peers { permanent_uuid: "adbbc0043cb14763a95172b87454443a" member_type: NON_VOTER last_known_addr { host: "127.13.109.60" port: 34827 } attrs { promote: true } } }
I20260811 19:00:27.016155 21027 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26408052dd7842e1a0a036cce7c682c6 [sys.catalog]: This master's current role is: LEARNER
I20260811 19:00:27.019922 20942 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 449f60a2c460491d97c80e5cd420ef6e: attempting to promote NON_VOTER 26408052dd7842e1a0a036cce7c682c6 to VOTER
I20260811 19:00:27.020509 20942 consensus_queue.cc:237] T 00000000000000000000000000000000 P 449f60a2c460491d97c80e5cd420ef6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "449f60a2c460491d97c80e5cd420ef6e" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 42815 } } peers { permanent_uuid: "26408052dd7842e1a0a036cce7c682c6" member_type: VOTER last_known_addr { host: "127.13.109.61" port: 43661 } attrs { promote: false } } peers { permanent_uuid: "adbbc0043cb14763a95172b87454443a" member_type: NON_VOTER last_known_addr { host: "127.13.109.60" port: 34827 } attrs { promote: true } }
W20260811 19:00:27.023002 20326 consensus_peers.cc:487] T 00000000000000000000000000000000 P 449f60a2c460491d97c80e5cd420ef6e -> Peer adbbc0043cb14763a95172b87454443a (127.13.109.60:34827): Couldn't send request to peer adbbc0043cb14763a95172b87454443a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260811 19:00:27.023752 20995 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 26408052dd7842e1a0a036cce7c682c6 [term 1 LEARNER]: Refusing update from remote peer 449f60a2c460491d97c80e5cd420ef6e: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260811 19:00:27.023953 21120 raft_consensus.cc:482] T 00000000000000000000000000000000 P adbbc0043cb14763a95172b87454443a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:00:27.024130 21120 raft_consensus.cc:504] T 00000000000000000000000000000000 P adbbc0043cb14763a95172b87454443a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "449f60a2c460491d97c80e5cd420ef6e" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 42815 } } peers { permanent_uuid: "26408052dd7842e1a0a036cce7c682c6" member_type: VOTER last_known_addr { host: "127.13.109.61" port: 43661 } } peers { permanent_uuid: "adbbc0043cb14763a95172b87454443a" member_type: VOTER last_known_addr { host: "127.13.109.60" port: 34827 } }
I20260811 19:00:27.024137 20942 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 449f60a2c460491d97c80e5cd420ef6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "26408052dd7842e1a0a036cce7c682c6" member_type: VOTER last_known_addr { host: "127.13.109.61" port: 43661 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260811 19:00:27.025077 21120 leader_election.cc:288] T 00000000000000000000000000000000 P adbbc0043cb14763a95172b87454443a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 449f60a2c460491d97c80e5cd420ef6e (127.13.109.62:42815), 26408052dd7842e1a0a036cce7c682c6 (127.13.109.61:43661)
I20260811 19:00:27.025573 20355 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "adbbc0043cb14763a95172b87454443a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "449f60a2c460491d97c80e5cd420ef6e" is_pre_election: true
I20260811 19:00:27.025564 20995 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "adbbc0043cb14763a95172b87454443a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "26408052dd7842e1a0a036cce7c682c6" is_pre_election: true
I20260811 19:00:27.026248 21057 leader_election.cc:302] T 00000000000000000000000000000000 P adbbc0043cb14763a95172b87454443a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: adbbc0043cb14763a95172b87454443a; no voters: 26408052dd7842e1a0a036cce7c682c6, 449f60a2c460491d97c80e5cd420ef6e
I20260811 19:00:27.026528 21120 raft_consensus.cc:2726] T 00000000000000000000000000000000 P adbbc0043cb14763a95172b87454443a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 19:00:27.026806 20949 raft_consensus.cc:2229] T 00000000000000000000000000000000 P adbbc0043cb14763a95172b87454443a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:00:27.027237 20949 raft_consensus.cc:2258] T 00000000000000000000000000000000 P adbbc0043cb14763a95172b87454443a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:00:27.027362 20949 tablet_replica.cc:330] T 00000000000000000000000000000000 P adbbc0043cb14763a95172b87454443a: stopping tablet replica
I20260811 19:00:27.029188 20942 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 449f60a2c460491d97c80e5cd420ef6e [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 26408052dd7842e1a0a036cce7c682c6 (127.13.109.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "449f60a2c460491d97c80e5cd420ef6e" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 42815 } } peers { permanent_uuid: "26408052dd7842e1a0a036cce7c682c6" member_type: VOTER last_known_addr { host: "127.13.109.61" port: 43661 } attrs { promote: false } } peers { permanent_uuid: "adbbc0043cb14763a95172b87454443a" member_type: NON_VOTER last_known_addr { host: "127.13.109.60" port: 34827 } attrs { promote: true } } }
I20260811 19:00:27.031000 20942 sys_catalog.cc:455] T 00000000000000000000000000000000 P 449f60a2c460491d97c80e5cd420ef6e [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "449f60a2c460491d97c80e5cd420ef6e" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "449f60a2c460491d97c80e5cd420ef6e" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 42815 } } peers { permanent_uuid: "26408052dd7842e1a0a036cce7c682c6" member_type: VOTER last_known_addr { host: "127.13.109.61" port: 43661 } attrs { promote: false } } peers { permanent_uuid: "adbbc0043cb14763a95172b87454443a" member_type: NON_VOTER last_known_addr { host: "127.13.109.60" port: 34827 } attrs { promote: true } } }
I20260811 19:00:27.031133 20942 sys_catalog.cc:458] T 00000000000000000000000000000000 P 449f60a2c460491d97c80e5cd420ef6e [sys.catalog]: This master's current role is: LEADER
I20260811 19:00:27.031739 20941 sys_catalog.cc:455] T 00000000000000000000000000000000 P 449f60a2c460491d97c80e5cd420ef6e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "449f60a2c460491d97c80e5cd420ef6e" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "449f60a2c460491d97c80e5cd420ef6e" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 42815 } } peers { permanent_uuid: "26408052dd7842e1a0a036cce7c682c6" member_type: VOTER last_known_addr { host: "127.13.109.61" port: 43661 } attrs { promote: false } } peers { permanent_uuid: "adbbc0043cb14763a95172b87454443a" member_type: NON_VOTER last_known_addr { host: "127.13.109.60" port: 34827 } attrs { promote: true } } }
I20260811 19:00:27.031848 20941 sys_catalog.cc:458] T 00000000000000000000000000000000 P 449f60a2c460491d97c80e5cd420ef6e [sys.catalog]: This master's current role is: LEADER
I20260811 19:00:27.032955 20993 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 26408052dd7842e1a0a036cce7c682c6 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 26408052dd7842e1a0a036cce7c682c6 (127.13.109.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "449f60a2c460491d97c80e5cd420ef6e" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 42815 } } peers { permanent_uuid: "26408052dd7842e1a0a036cce7c682c6" member_type: VOTER last_known_addr { host: "127.13.109.61" port: 43661 } attrs { promote: false } } peers { permanent_uuid: "adbbc0043cb14763a95172b87454443a" member_type: NON_VOTER last_known_addr { host: "127.13.109.60" port: 34827 } attrs { promote: true } } }
I20260811 19:00:27.034888 20949 master.cc:552] Master@127.13.109.60:34827 shutdown complete.
I20260811 19:00:27.034976 20949 master_runner.cc:305] Clearing existing system tablet
I20260811 19:00:27.034986 21145 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26408052dd7842e1a0a036cce7c682c6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "449f60a2c460491d97c80e5cd420ef6e" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "449f60a2c460491d97c80e5cd420ef6e" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 42815 } } peers { permanent_uuid: "26408052dd7842e1a0a036cce7c682c6" member_type: VOTER last_known_addr { host: "127.13.109.61" port: 43661 } attrs { promote: false } } peers { permanent_uuid: "adbbc0043cb14763a95172b87454443a" member_type: NON_VOTER last_known_addr { host: "127.13.109.60" port: 34827 } attrs { promote: true } } }
I20260811 19:00:27.035125 21145 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26408052dd7842e1a0a036cce7c682c6 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 19:00:27.035566 20949 fs_manager.cc:359] Metadata directory not provided
I20260811 19:00:27.035634 20949 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:00:27.037045 20949 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 19:00:27.038039 21149 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:27.038271 20949 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:00:27.038396 20949 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 19:00:27.038446 20949 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/master-2/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/master-2/wal
uuid: "adbbc0043cb14763a95172b87454443a"
format_stamp: "Formatted at 2026-08-11 19:00:26 on dist-test-slave-0kls"
server_key: "5831d0fd79fcd0c89385005d22cbea15"
server_key_iv: "c25aab5965f2462b552c9b3bcdb9a7b3"
server_key_version: "encryptionkey@0"
I20260811 19:00:27.041018 20949 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P adbbc0043cb14763a95172b87454443a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:00:27.043552 20949 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P adbbc0043cb14763a95172b87454443a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 19:00:27.043649 20949 log.cc:1196] T 00000000000000000000000000000000 P adbbc0043cb14763a95172b87454443a: Deleting WAL directory at /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260811 19:00:27.043905 20949 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P adbbc0043cb14763a95172b87454443a: Deleting consensus metadata
I20260811 19:00:27.044108 20949 master_runner.cc:315] Copying system tablet from 127.13.109.62:42815
I20260811 19:00:27.045339 20949 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P adbbc0043cb14763a95172b87454443a: tablet copy: Beginning tablet copy session from remote peer at address 127.13.109.62:42815
I20260811 19:00:27.045518 20692 heartbeater.cc:503] Master 127.13.109.62:42815 was elected leader, sending a full tablet report...
I20260811 19:00:27.056123 20386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:27.046420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50206 (local address 127.13.109.62:42815)
0811 19:00:27.046571 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:27.046576 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:27.047240 (+ 664us) server_negotiation.cc:407] Connection header received
0811 19:00:27.047349 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:27.047351 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:27.047401 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:27.047487 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:27.048069 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:27.048990 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:27.050259 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:27.050630 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:27.050737 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:27.052072 (+ 1335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:27.052077 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:27.052080 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:27.052107 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:27.054293 (+ 2186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:27.054783 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:27.054787 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:27.054789 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:27.054879 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:27.055192 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:27.055195 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:27.055197 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:27.055426 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:27.055535 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:27.055832 (+ 297us) server_negotiation.cc:299] Negotiation successful
0811 19:00:27.055958 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 19:00:27.056119 21156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:27.046037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:42815 (local address 127.0.0.1:50206)
0811 19:00:27.047137 (+ 1100us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:27.047153 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:27.047263 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:27.047527 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:27.047530 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:27.047770 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:27.047916 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:27.047923 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:27.049113 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:27.049116 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:27.050133 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:27.050145 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:27.050770 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:27.050774 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:27.050868 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:27.051451 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:27.051466 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:27.051918 (+ 452us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:00:27.054404 (+ 2486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:27.054408 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:27.054410 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:27.054675 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:27.054980 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:27.054983 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:27.054985 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:27.055092 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:27.055531 (+ 439us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:00:27.055534 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:00:27.055645 (+ 111us) client_negotiation.cc:769] Sending connection context
0811 19:00:27.055757 (+ 112us) client_negotiation.cc:240] Negotiation successful
0811 19:00:27.055915 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1026,"thread_start_us":91,"threads_started":1}
I20260811 19:00:27.056905 20365 tablet_copy_service.cc:140] P 449f60a2c460491d97c80e5cd420ef6e: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer adbbc0043cb14763a95172b87454443a ({username='oryx', principal='oryx/127.13.109.60@KRBTEST.COM'} at 127.0.0.1:50206)
I20260811 19:00:27.057006 20365 tablet_copy_service.cc:161] P 449f60a2c460491d97c80e5cd420ef6e: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer adbbc0043cb14763a95172b87454443a at {username='oryx', principal='oryx/127.13.109.60@KRBTEST.COM'} at 127.0.0.1:50206: session id = adbbc0043cb14763a95172b87454443a-00000000000000000000000000000000
I20260811 19:00:27.058178 20365 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 449f60a2c460491d97c80e5cd420ef6e: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 19:00:27.059720 20949 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260811 19:00:27.062302 20949 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P adbbc0043cb14763a95172b87454443a: tablet copy: Starting download of 0 data blocks...
I20260811 19:00:27.062503 20949 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P adbbc0043cb14763a95172b87454443a: tablet copy: Starting download of 1 WAL segments...
I20260811 19:00:27.064100 20949 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P adbbc0043cb14763a95172b87454443a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 19:00:27.066053 20365 tablet_copy_service.cc:342] P 449f60a2c460491d97c80e5cd420ef6e: Request end of tablet copy session adbbc0043cb14763a95172b87454443a-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.13.109.60@KRBTEST.COM'} at 127.0.0.1:50206
I20260811 19:00:27.066148 20365 tablet_copy_service.cc:433] P 449f60a2c460491d97c80e5cd420ef6e: ending tablet copy session adbbc0043cb14763a95172b87454443a-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer adbbc0043cb14763a95172b87454443a
I20260811 19:00:27.073832 20949 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:00:27.075657 21157 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:27.075730 21160 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:27.075963 21158 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:27.076421 20949 server_base.cc:1034] running on GCE node
I20260811 19:00:27.076587 20949 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:00:27.076905 20949 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:00:27.078079 20949 hybrid_clock.cc:648] HybridClock initialized: now 1786474827078061 us; error 48 us; skew 500 ppm
Aug 11 19:00:27 dist-test-slave-0kls krb5kdc[20293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474827, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.109.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:00:27.079777 20949 init.cc:377] Logged in from keytab as oryx/127.13.109.60@KRBTEST.COM (short username oryx)
I20260811 19:00:27.080255 20949 webserver.cc:460] Webserver started at http://127.13.109.60:39055/ using document root <none> and password file <none>
I20260811 19:00:27.080446 20949 fs_manager.cc:359] Metadata directory not provided
I20260811 19:00:27.080487 20949 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:00:27.081881 20949 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 19:00:27.082976 21169 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:27.083288 20949 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 19:00:27.083375 20949 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/master-2/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/master-2/wal
uuid: "adbbc0043cb14763a95172b87454443a"
format_stamp: "Formatted at 2026-08-11 19:00:26 on dist-test-slave-0kls"
server_key: "5831d0fd79fcd0c89385005d22cbea15"
server_key_iv: "c25aab5965f2462b552c9b3bcdb9a7b3"
server_key_version: "encryptionkey@0"
I20260811 19:00:27.083467 20949 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:00:27.097133 20949 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:00:27.099668 20949 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:00:27.104964 20949 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.60:34827
I20260811 19:00:27.105001 21224 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.60:34827 every 8 connection(s)
I20260811 19:00:27.105612 20949 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/master-2/data/info.pb
I20260811 19:00:27.107820 21225 sys_catalog.cc:263] Verifying existing consensus state
I20260811 19:00:27.109019 21225 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P adbbc0043cb14763a95172b87454443a: Bootstrap starting.
I20260811 19:00:27.120802 21225 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P adbbc0043cb14763a95172b87454443a: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 19:00:27.121133 21225 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P adbbc0043cb14763a95172b87454443a: Bootstrap complete.
I20260811 19:00:27.121730 21225 raft_consensus.cc:348] T 00000000000000000000000000000000 P adbbc0043cb14763a95172b87454443a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "449f60a2c460491d97c80e5cd420ef6e" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 42815 } } peers { permanent_uuid: "26408052dd7842e1a0a036cce7c682c6" member_type: VOTER last_known_addr { host: "127.13.109.61" port: 43661 } attrs { promote: false } } peers { permanent_uuid: "adbbc0043cb14763a95172b87454443a" member_type: NON_VOTER last_known_addr { host: "127.13.109.60" port: 34827 } attrs { promote: true } }
I20260811 19:00:27.121906 21225 raft_consensus.cc:729] T 00000000000000000000000000000000 P adbbc0043cb14763a95172b87454443a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: adbbc0043cb14763a95172b87454443a, State: Initialized, Role: LEARNER
I20260811 19:00:27.122031 21225 consensus_queue.cc:260] T 00000000000000000000000000000000 P adbbc0043cb14763a95172b87454443a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "449f60a2c460491d97c80e5cd420ef6e" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 42815 } } peers { permanent_uuid: "26408052dd7842e1a0a036cce7c682c6" member_type: VOTER last_known_addr { host: "127.13.109.61" port: 43661 } attrs { promote: false } } peers { permanent_uuid: "adbbc0043cb14763a95172b87454443a" member_type: NON_VOTER last_known_addr { host: "127.13.109.60" port: 34827 } attrs { promote: true } }
I20260811 19:00:27.122473 21229 sys_catalog.cc:455] T 00000000000000000000000000000000 P adbbc0043cb14763a95172b87454443a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "449f60a2c460491d97c80e5cd420ef6e" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 42815 } } peers { permanent_uuid: "26408052dd7842e1a0a036cce7c682c6" member_type: VOTER last_known_addr { host: "127.13.109.61" port: 43661 } attrs { promote: false } } peers { permanent_uuid: "adbbc0043cb14763a95172b87454443a" member_type: NON_VOTER last_known_addr { host: "127.13.109.60" port: 34827 } attrs { promote: true } } }
I20260811 19:00:27.122632 21225 sys_catalog.cc:564] T 00000000000000000000000000000000 P adbbc0043cb14763a95172b87454443a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:00:27.122707 21229 sys_catalog.cc:458] T 00000000000000000000000000000000 P adbbc0043cb14763a95172b87454443a [sys.catalog]: This master's current role is: LEARNER
I20260811 19:00:27.126893 21240 catalog_manager.cc:1259] Loaded cluster ID: 2362e23819bd411a8fb9f457d6f232c5
I20260811 19:00:27.126967 21240 catalog_manager.cc:1552] T 00000000000000000000000000000000 P adbbc0043cb14763a95172b87454443a: loading cluster ID for follower catalog manager: success
I20260811 19:00:27.129164 21240 catalog_manager.cc:1574] T 00000000000000000000000000000000 P adbbc0043cb14763a95172b87454443a: acquiring CA information for follower catalog manager: success
I20260811 19:00:27.130695 21240 catalog_manager.cc:1602] T 00000000000000000000000000000000 P adbbc0043cb14763a95172b87454443a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260811 19:00:27.133711 21143 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:00:27.134057 21143 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:00:27.134133 21143 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:00:27.138590 21143 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:00:27.138641 21143 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:00:27.138728 21143 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.1
I20260811 19:00:27.144172 21143 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:36645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/krb5kdc/oryx_127.13.109.1.keytab
--principal=oryx/127.13.109.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.1:38457
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.109.1
--webserver_port=36063
--tserver_master_addrs=127.13.109.62:42815,127.13.109.61:43661,127.13.109.60:34827
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21143
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.1
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:00:27.145341 21143 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:00:27.146497 21143 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:00:27.157028 21245 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:27.157028 21248 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:27.157117 21246 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:27.157366 21143 server_base.cc:1034] running on GCE node
I20260811 19:00:27.157975 21143 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:00:27.158667 21143 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:00:27.159876 21143 hybrid_clock.cc:648] HybridClock initialized: now 1786474827159857 us; error 30 us; skew 500 ppm
Aug 11 19:00:27 dist-test-slave-0kls krb5kdc[20293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474827, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.109.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:00:27.162830 21143 init.cc:377] Logged in from keytab as oryx/127.13.109.1@KRBTEST.COM (short username oryx)
I20260811 19:00:27.163729 21143 webserver.cc:460] Webserver started at http://127.13.109.1:36063/ using document root <none> and password file <none>
I20260811 19:00:27.164322 21143 fs_manager.cc:359] Metadata directory not provided
I20260811 19:00:27.164443 21143 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:00:27.168613 21143 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 19:00:27.170939 21255 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:27.172328 21143 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 19:00:27.172497 21143 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/ts-0/wal
uuid: "2c9427fe9258493f9465751fef6bb43a"
format_stamp: "Formatted at 2026-08-11 19:00:25 on dist-test-slave-0kls"
server_key: "6ebeeb2e88231fd5c41e982cc1f7a1f1"
server_key_iv: "f1be4741d75737dd30357c028c0172aa"
server_key_version: "encryptionkey@0"
I20260811 19:00:27.173127 21143 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:00:27.191145 21143 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:00:27.194805 21143 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:00:27.195065 21143 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:00:27.195703 21143 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:00:27.196648 21143 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:00:27.196720 21143 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:27.196805 21143 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:00:27.196854 21143 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:27.207105 21143 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.1:38457
I20260811 19:00:27.207207 21375 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.1:38457 every 8 connection(s)
I20260811 19:00:27.208226 21143 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
I20260811 19:00:27.214561 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 21143
I20260811 19:00:27.214764 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 20554
Aug 11 19:00:27 dist-test-slave-0kls krb5kdc[20293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474827, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.109.1@KRBTEST.COM for oryx/127.13.109.61@KRBTEST.COM
Aug 11 19:00:27 dist-test-slave-0kls krb5kdc[20293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474827, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.109.1@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM
Aug 11 19:00:27 dist-test-slave-0kls krb5kdc[20293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474827, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.109.1@KRBTEST.COM for oryx/127.13.109.60@KRBTEST.COM
I20260811 19:00:27.226534 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.2:32839
--local_ip_for_outbound_sockets=127.13.109.2
--tserver_master_addrs=127.13.109.62:42815
--webserver_port=45991
--webserver_interface=127.13.109.2
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/krb5kdc/oryx_127.13.109.2.keytab
--principal=oryx/127.13.109.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:36645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.13.109.62:42815,127.13.109.61:43661
--tserver_master_addrs=127.13.109.62:42815,127.13.109.61:43661,127.13.109.60:34827 with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 19:00:27.227478 20386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:27.210196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.1:55023 (local address 127.13.109.62:42815)
0811 19:00:27.210366 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:27.210371 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:27.211331 (+ 960us) server_negotiation.cc:407] Connection header received
0811 19:00:27.212492 (+ 1161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:27.212495 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:27.212559 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:27.212664 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:27.214454 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:27.215274 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:27.218094 (+ 2820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:27.218422 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:27.218512 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:27.222310 (+ 3798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:27.222314 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:27.222317 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:27.222348 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:27.224675 (+ 2327us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:27.225580 (+ 905us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:27.225584 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:27.225586 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:27.225681 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:27.226058 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:27.226061 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:27.226063 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:27.226313 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:27.226401 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:27.227155 (+ 754us) server_negotiation.cc:299] Negotiation successful
0811 19:00:27.227292 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 19:00:27.228652 21385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:27.211755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.60:34827 (local address 127.13.109.1:56773)
0811 19:00:27.212000 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:27.212020 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:27.212297 (+ 277us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:27.212870 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:27.212872 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:27.213482 (+ 610us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:27.214293 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:27.214306 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:27.215632 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:27.215638 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:27.217287 (+ 1649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:27.217298 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:27.218091 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:27.218096 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:27.218295 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:27.219389 (+ 1094us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:27.219407 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:27.222974 (+ 3567us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:00:27.226229 (+ 3255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:27.226232 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:27.226234 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:27.226486 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:27.226817 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:27.226819 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:27.226821 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:27.226905 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:27.227403 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:00:27.227406 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:00:27.227492 (+ 86us) client_negotiation.cc:769] Sending connection context
0811 19:00:27.227597 (+ 105us) client_negotiation.cc:240] Negotiation successful
0811 19:00:27.227782 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"mutex_wait_us":231,"spinlock_wait_cycles":3968,"thread_start_us":82,"threads_started":1}
I20260811 19:00:27.228652 21381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:27.210676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:42815 (local address 127.13.109.1:55023)
0811 19:00:27.211165 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:27.211221 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:27.212285 (+ 1064us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:27.212845 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:27.212852 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:27.213482 (+ 630us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:27.214303 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:27.214310 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:27.215456 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:27.215459 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:27.217955 (+ 2496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:27.217965 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:27.218723 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:27.218727 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:27.218832 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:27.219274 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:27.219292 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:27.222124 (+ 2832us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:00:27.225144 (+ 3020us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:27.225148 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:27.225152 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:27.225430 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:27.225832 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:27.225838 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:27.225841 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:27.225940 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:27.226465 (+ 525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:00:27.226468 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:00:27.226937 (+ 469us) client_negotiation.cc:769] Sending connection context
0811 19:00:27.227921 (+ 984us) client_negotiation.cc:240] Negotiation successful
0811 19:00:27.228052 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"mutex_wait_us":1863,"spinlock_wait_cycles":37376,"thread_start_us":146,"threads_started":1}
I20260811 19:00:27.229410 21118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:27.210205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.1:33851 (local address 127.13.109.61:43661)
0811 19:00:27.210356 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:27.210360 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:27.211331 (+ 971us) server_negotiation.cc:407] Connection header received
0811 19:00:27.212602 (+ 1271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:27.212605 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:27.212660 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:27.212734 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:27.214438 (+ 1704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:27.215280 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:27.217432 (+ 2152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:27.217881 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:27.217996 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:27.221854 (+ 3858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:27.221861 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:27.221868 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:27.221905 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:27.223896 (+ 1991us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:27.225107 (+ 1211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:27.225114 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:27.225118 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:27.225222 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:27.225698 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:27.225701 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:27.225703 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:27.226171 (+ 468us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:27.226275 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:27.229113 (+ 2838us) server_negotiation.cc:299] Negotiation successful
0811 19:00:27.229265 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 19:00:27.228948 21382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:27.210675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.61:43661 (local address 127.13.109.1:33851)
0811 19:00:27.211165 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:27.211222 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:27.212300 (+ 1078us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:27.212869 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:27.212872 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:27.213482 (+ 610us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:27.214245 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:27.214267 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:27.215459 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:27.215466 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:27.217278 (+ 1812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:27.217293 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:27.218159 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:27.218166 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:27.218295 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:27.219111 (+ 816us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:27.219139 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:27.221640 (+ 2501us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:00:27.224016 (+ 2376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:27.224026 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:27.224061 (+ 35us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:27.224638 (+ 577us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:27.225343 (+ 705us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:27.225349 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:27.225354 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:27.225580 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:27.226337 (+ 757us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:00:27.226345 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:00:27.226757 (+ 412us) client_negotiation.cc:769] Sending connection context
0811 19:00:27.227004 (+ 247us) client_negotiation.cc:240] Negotiation successful
0811 19:00:27.227431 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"mutex_wait_us":207,"spinlock_wait_cycles":122112,"thread_start_us":154,"threads_started":1}
I20260811 19:00:27.229626 21386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:27.211901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.1:56773 (local address 127.13.109.60:34827)
0811 19:00:27.212199 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:27.212211 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:27.212235 (+ 24us) server_negotiation.cc:407] Connection header received
0811 19:00:27.212595 (+ 360us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:27.212598 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:27.212655 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:27.212734 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:27.214449 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:27.215510 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:27.217428 (+ 1918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:27.217803 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:27.217901 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:27.223314 (+ 5413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:27.223323 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:27.223328 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:27.223364 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:27.226103 (+ 2739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:27.226599 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:27.226603 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:27.226605 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:27.226711 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:27.227012 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:27.227018 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:27.227023 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:27.227267 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:27.227357 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:27.229280 (+ 1923us) server_negotiation.cc:299] Negotiation successful
0811 19:00:27.229427 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":217,"thread_start_us":110,"threads_started":1}
I20260811 19:00:27.230280 21376 heartbeater.cc:347] Connected to a master server at 127.13.109.60:34827
I20260811 19:00:27.230559 21376 heartbeater.cc:464] Registering TS with master...
I20260811 19:00:27.230664 21378 heartbeater.cc:347] Connected to a master server at 127.13.109.61:43661
I20260811 19:00:27.230742 21378 heartbeater.cc:464] Registering TS with master...
I20260811 19:00:27.231204 21376 heartbeater.cc:511] Master 127.13.109.60:34827 requested a full tablet report, sending...
I20260811 19:00:27.230280 21377 heartbeater.cc:347] Connected to a master server at 127.13.109.62:42815
I20260811 19:00:27.231765 21377 heartbeater.cc:464] Registering TS with master...
I20260811 19:00:27.231930 21377 heartbeater.cc:511] Master 127.13.109.62:42815 requested a full tablet report, sending...
I20260811 19:00:27.232195 21378 heartbeater.cc:511] Master 127.13.109.61:43661 requested a full tablet report, sending...
I20260811 19:00:27.232818 20343 ts_manager.cc:194] Re-registered known tserver with Master: 2c9427fe9258493f9465751fef6bb43a (127.13.109.1:38457)
I20260811 19:00:27.232914 21182 ts_manager.cc:194] Registered new tserver with Master: 2c9427fe9258493f9465751fef6bb43a (127.13.109.1:38457)
I20260811 19:00:27.233259 20983 ts_manager.cc:194] Registered new tserver with Master: 2c9427fe9258493f9465751fef6bb43a (127.13.109.1:38457)
I20260811 19:00:27.233661 20343 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.109.1@KRBTEST.COM'} at 127.13.109.1:55023
I20260811 19:00:27.275833 20834 heartbeater.cc:503] Master 127.13.109.62:42815 was elected leader, sending a full tablet report...
W20260811 19:00:27.352248 21387 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:00:27.352515 21387 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:00:27.352607 21387 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:00:27.356685 21387 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:00:27.356752 21387 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:00:27.356865 21387 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.2
I20260811 19:00:27.362344 21387 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:36645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/krb5kdc/oryx_127.13.109.2.keytab
--principal=oryx/127.13.109.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.2:32839
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.109.2
--webserver_port=45991
--tserver_master_addrs=127.13.109.62:42815,127.13.109.61:43661,127.13.109.60:34827
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21387
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.2
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:00:27.363559 21387 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:00:27.364675 21387 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:00:27.374763 21394 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:27.374786 21393 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:27.374840 21387 server_base.cc:1034] running on GCE node
W20260811 19:00:27.374785 21396 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:27.375643 21387 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:00:27.376454 21387 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:00:27.377652 21387 hybrid_clock.cc:648] HybridClock initialized: now 1786474827377631 us; error 44 us; skew 500 ppm
Aug 11 19:00:27 dist-test-slave-0kls krb5kdc[20293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474827, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.109.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:00:27.380605 21387 init.cc:377] Logged in from keytab as oryx/127.13.109.2@KRBTEST.COM (short username oryx)
I20260811 19:00:27.381615 21387 webserver.cc:460] Webserver started at http://127.13.109.2:45991/ using document root <none> and password file <none>
I20260811 19:00:27.382331 21387 fs_manager.cc:359] Metadata directory not provided
I20260811 19:00:27.382419 21387 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:00:27.386803 21387 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 19:00:27.388933 21403 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:27.390300 21387 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 19:00:27.390420 21387 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/ts-1/wal
uuid: "adf5bc4ddc704fec824189012815251a"
format_stamp: "Formatted at 2026-08-11 19:00:25 on dist-test-slave-0kls"
server_key: "f45197dda9c23035b014f6bfd149109d"
server_key_iv: "4d5cab164c4d78df9448cd585c3d5b5c"
server_key_version: "encryptionkey@0"
I20260811 19:00:27.390975 21387 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:00:27.411607 21387 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:00:27.414960 21387 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:00:27.415190 21387 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:00:27.415936 21387 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:00:27.416852 21387 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:00:27.416927 21387 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:27.417003 21387 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:00:27.417057 21387 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:27.426599 21387 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.2:32839
I20260811 19:00:27.426654 21524 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.2:32839 every 8 connection(s)
I20260811 19:00:27.427687 21387 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
I20260811 19:00:27.428263 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 21387
I20260811 19:00:27.428460 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 20697
I20260811 19:00:27.442080 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.3:43545
--local_ip_for_outbound_sockets=127.13.109.3
--tserver_master_addrs=127.13.109.62:42815
--webserver_port=36667
--webserver_interface=127.13.109.3
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/krb5kdc/oryx_127.13.109.3.keytab
--principal=oryx/127.13.109.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:36645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.13.109.62:42815,127.13.109.61:43661
--tserver_master_addrs=127.13.109.62:42815,127.13.109.61:43661,127.13.109.60:34827 with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 19:00:27 dist-test-slave-0kls krb5kdc[20293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474827, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.109.2@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM
Aug 11 19:00:27 dist-test-slave-0kls krb5kdc[20293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474827, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.109.2@KRBTEST.COM for oryx/127.13.109.61@KRBTEST.COM
Aug 11 19:00:27 dist-test-slave-0kls krb5kdc[20293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474827, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.109.2@KRBTEST.COM for oryx/127.13.109.60@KRBTEST.COM
I20260811 19:00:27.452412 21118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:27.434485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.2:41613 (local address 127.13.109.61:43661)
0811 19:00:27.434648 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:27.434654 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:27.434919 (+ 265us) server_negotiation.cc:407] Connection header received
0811 19:00:27.435397 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:27.435403 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:27.435466 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:27.435533 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:27.437858 (+ 2325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:27.438692 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:27.441080 (+ 2388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:27.441440 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:27.441539 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:27.446685 (+ 5146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:27.446688 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:27.446691 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:27.446720 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:27.449110 (+ 2390us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:27.450160 (+ 1050us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:27.450165 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:27.450167 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:27.450282 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:27.450842 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:27.450846 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:27.450847 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:27.451119 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:27.451197 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:27.452098 (+ 901us) server_negotiation.cc:299] Negotiation successful
0811 19:00:27.452236 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 19:00:27.452412 20386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:27.430810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.2:42901 (local address 127.13.109.62:42815)
0811 19:00:27.430955 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:27.430959 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:27.433274 (+ 2315us) server_negotiation.cc:407] Connection header received
0811 19:00:27.435302 (+ 2028us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:27.435309 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:27.435378 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:27.435439 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:27.437852 (+ 2413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:27.438773 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:27.441078 (+ 2305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:27.441460 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:27.441558 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:27.446676 (+ 5118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:27.446680 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:27.446683 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:27.446718 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:27.449108 (+ 2390us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:27.450151 (+ 1043us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:27.450155 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:27.450157 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:27.450282 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:27.450842 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:27.450846 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:27.450847 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:27.451102 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:27.451196 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:27.452089 (+ 893us) server_negotiation.cc:299] Negotiation successful
0811 19:00:27.452236 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 19:00:27.452548 21386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:27.434724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.2:34383 (local address 127.13.109.60:34827)
0811 19:00:27.434857 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:27.434864 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:27.434882 (+ 18us) server_negotiation.cc:407] Connection header received
0811 19:00:27.435510 (+ 628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:27.435514 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:27.435570 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:27.435649 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:27.437850 (+ 2201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:27.438689 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:27.441089 (+ 2400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:27.441455 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:27.441529 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:27.446664 (+ 5135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:27.446672 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:27.446678 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:27.446718 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:27.449141 (+ 2423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:27.450194 (+ 1053us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:27.450200 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:27.450203 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:27.450323 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:27.450871 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:27.450878 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:27.450881 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:27.451185 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:27.451289 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:27.452127 (+ 838us) server_negotiation.cc:299] Negotiation successful
0811 19:00:27.452310 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 19:00:27.454751 21531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:27.434372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.61:43661 (local address 127.13.109.2:41613)
0811 19:00:27.434815 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:27.434827 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:27.435024 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:27.435833 (+ 809us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:27.435846 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:27.436690 (+ 844us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:27.437619 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:27.437644 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:27.438900 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:27.438904 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:27.440932 (+ 2028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:27.440938 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:27.441769 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:27.441773 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:27.442455 (+ 682us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:27.443109 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:27.443126 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:27.446400 (+ 3274us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:00:27.449322 (+ 2922us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:27.449331 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:27.449355 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:27.450001 (+ 646us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:27.450452 (+ 451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:27.450455 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:27.450457 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:27.450716 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:27.451274 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:00:27.451282 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:00:27.451758 (+ 476us) client_negotiation.cc:769] Sending connection context
0811 19:00:27.452853 (+ 1095us) client_negotiation.cc:240] Negotiation successful
0811 19:00:27.453482 (+ 629us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":384,"mutex_wait_us":1091,"spinlock_wait_cycles":60800,"thread_start_us":74,"threads_started":1}
I20260811 19:00:27.454756 21530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:27.433846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.60:34827 (local address 127.13.109.2:34383)
0811 19:00:27.434553 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:27.434569 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:27.435094 (+ 525us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:27.435953 (+ 859us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:27.435955 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:27.436686 (+ 731us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:27.437619 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:27.437644 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:27.438900 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:27.438904 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:27.440916 (+ 2012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:27.440931 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:27.441769 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:27.441773 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:27.442197 (+ 424us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:27.442905 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:27.442923 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:27.446400 (+ 3477us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:00:27.449322 (+ 2922us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:27.449332 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:27.449357 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:27.450033 (+ 676us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:27.450460 (+ 427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:27.450465 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:27.450469 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:27.450731 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:27.451364 (+ 633us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:00:27.451369 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:00:27.451766 (+ 397us) client_negotiation.cc:769] Sending connection context
0811 19:00:27.453090 (+ 1324us) client_negotiation.cc:240] Negotiation successful
0811 19:00:27.453421 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":629,"mutex_wait_us":835,"spinlock_wait_cycles":205440,"thread_start_us":105,"threads_started":1}
I20260811 19:00:27.455034 21529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:27.431612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:42815 (local address 127.13.109.2:42901)
0811 19:00:27.433095 (+ 1483us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:27.433143 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:27.434991 (+ 1848us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:27.435833 (+ 842us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:27.435846 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:27.436718 (+ 872us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:27.437619 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:27.437645 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:27.438940 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:27.438946 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:27.440925 (+ 1979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:27.440941 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:27.441776 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:27.441805 (+ 29us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:27.442154 (+ 349us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:27.442977 (+ 823us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:27.442998 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:27.446400 (+ 3402us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:00:27.449322 (+ 2922us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:27.449331 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:27.449355 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:27.450001 (+ 646us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:27.450452 (+ 451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:27.450455 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:27.450457 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:27.450716 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:27.451274 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:00:27.451282 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:00:27.451758 (+ 476us) client_negotiation.cc:769] Sending connection context
0811 19:00:27.452765 (+ 1007us) client_negotiation.cc:240] Negotiation successful
0811 19:00:27.453295 (+ 530us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1252,"mutex_wait_us":573,"spinlock_wait_cycles":46848,"thread_start_us":1048,"threads_started":1}
I20260811 19:00:27.456426 21527 heartbeater.cc:347] Connected to a master server at 127.13.109.61:43661
I20260811 19:00:27.456605 21526 heartbeater.cc:347] Connected to a master server at 127.13.109.62:42815
I20260811 19:00:27.456722 21527 heartbeater.cc:464] Registering TS with master...
I20260811 19:00:27.456948 21525 heartbeater.cc:347] Connected to a master server at 127.13.109.60:34827
I20260811 19:00:27.457033 21525 heartbeater.cc:464] Registering TS with master...
I20260811 19:00:27.457403 21527 heartbeater.cc:511] Master 127.13.109.61:43661 requested a full tablet report, sending...
I20260811 19:00:27.457401 21525 heartbeater.cc:511] Master 127.13.109.60:34827 requested a full tablet report, sending...
I20260811 19:00:27.456728 21526 heartbeater.cc:464] Registering TS with master...
I20260811 19:00:27.458244 21526 heartbeater.cc:511] Master 127.13.109.62:42815 requested a full tablet report, sending...
I20260811 19:00:27.459128 20983 ts_manager.cc:194] Registered new tserver with Master: adf5bc4ddc704fec824189012815251a (127.13.109.2:32839)
I20260811 19:00:27.459206 20343 ts_manager.cc:194] Re-registered known tserver with Master: adf5bc4ddc704fec824189012815251a (127.13.109.2:32839)
I20260811 19:00:27.459676 21182 ts_manager.cc:194] Registered new tserver with Master: adf5bc4ddc704fec824189012815251a (127.13.109.2:32839)
I20260811 19:00:27.460005 20343 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.109.2@KRBTEST.COM'} at 127.13.109.2:42901
W20260811 19:00:27.572464 21532 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:00:27.572721 21532 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:00:27.572805 21532 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:00:27.576826 21532 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:00:27.576889 21532 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:00:27.576949 21532 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.3
I20260811 19:00:27.582628 21532 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:36645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/krb5kdc/oryx_127.13.109.3.keytab
--principal=oryx/127.13.109.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.3:43545
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.109.3
--webserver_port=36667
--tserver_master_addrs=127.13.109.62:42815,127.13.109.61:43661,127.13.109.60:34827
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21532
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.3
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:00:27.583863 21532 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:00:27.584939 21532 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:00:27.595314 21540 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:27.595305 21542 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:27.595305 21539 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:27.595784 21532 server_base.cc:1034] running on GCE node
I20260811 19:00:27.596306 21532 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:00:27.597062 21532 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:00:27.598253 21532 hybrid_clock.cc:648] HybridClock initialized: now 1786474827598216 us; error 48 us; skew 500 ppm
Aug 11 19:00:27 dist-test-slave-0kls krb5kdc[20293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474827, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.109.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:00:27.601127 21532 init.cc:377] Logged in from keytab as oryx/127.13.109.3@KRBTEST.COM (short username oryx)
I20260811 19:00:27.602182 21532 webserver.cc:460] Webserver started at http://127.13.109.3:36667/ using document root <none> and password file <none>
I20260811 19:00:27.602820 21532 fs_manager.cc:359] Metadata directory not provided
I20260811 19:00:27.602924 21532 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:00:27.606950 21532 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 19:00:27.609200 21549 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:27.610723 21532 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 19:00:27.610852 21532 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/ts-2/wal
uuid: "20f4f9eaa6844d8b9466235cf97ec8ce"
format_stamp: "Formatted at 2026-08-11 19:00:26 on dist-test-slave-0kls"
server_key: "1d09b557eaa214599a885872d16287fc"
server_key_iv: "9b1bb1153e3e2c6ee87d2b546e6b366c"
server_key_version: "encryptionkey@0"
I20260811 19:00:27.611406 21532 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:00:27.617080 21550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:27.612460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.60:34827 (local address 127.0.0.1:46550)
0811 19:00:27.612719 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:27.612733 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:27.612843 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:27.613172 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:27.613174 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:27.613190 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:00:27.613282 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:27.613292 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:27.614338 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:27.614344 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:27.615745 (+ 1401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:27.615757 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:27.616706 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:27.616710 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:27.616795 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:27.616819 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 19:00:27.616873 (+ 54us) client_negotiation.cc:240] Negotiation successful
0811 19:00:27.616936 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":78,"threads_started":1}
I20260811 19:00:27.617100 21386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:27.612549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46550 (local address 127.13.109.60:34827)
0811 19:00:27.612786 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:27.612791 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:27.612811 (+ 20us) server_negotiation.cc:407] Connection header received
0811 19:00:27.612975 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:27.612978 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:27.613050 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:27.613153 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:00:27.613390 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:27.614195 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:27.615902 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:27.616571 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:27.616690 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:27.616857 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:27.616900 (+ 43us) server_negotiation.cc:299] Negotiation successful
0811 19:00:27.616971 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142}
I20260811 19:00:27.617983 21193 raft_consensus.cc:1203] T 00000000000000000000000000000000 P adbbc0043cb14763a95172b87454443a [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260811 19:00:27.618429 21229 sys_catalog.cc:455] T 00000000000000000000000000000000 P adbbc0043cb14763a95172b87454443a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 449f60a2c460491d97c80e5cd420ef6e. Latest consensus state: current_term: 1 leader_uuid: "449f60a2c460491d97c80e5cd420ef6e" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "449f60a2c460491d97c80e5cd420ef6e" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 42815 } } peers { permanent_uuid: "26408052dd7842e1a0a036cce7c682c6" member_type: VOTER last_known_addr { host: "127.13.109.61" port: 43661 } attrs { promote: false } } peers { permanent_uuid: "adbbc0043cb14763a95172b87454443a" member_type: NON_VOTER last_known_addr { host: "127.13.109.60" port: 34827 } attrs { promote: true } } }
I20260811 19:00:27.618587 21229 sys_catalog.cc:458] T 00000000000000000000000000000000 P adbbc0043cb14763a95172b87454443a [sys.catalog]: This master's current role is: LEARNER
I20260811 19:00:27.630681 21532 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:00:27.633903 21532 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:00:27.634063 21532 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:00:27.634799 21532 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:00:27.635749 21532 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:00:27.635798 21532 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:27.635840 21532 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:00:27.635855 21532 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:27.645594 21532 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.3:43545
I20260811 19:00:27.645618 21676 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.3:43545 every 8 connection(s)
I20260811 19:00:27.646661 21532 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
I20260811 19:00:27.651638 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 21532
Aug 11 19:00:27 dist-test-slave-0kls krb5kdc[20293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474827, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.109.3@KRBTEST.COM for oryx/127.13.109.61@KRBTEST.COM
Aug 11 19:00:27 dist-test-slave-0kls krb5kdc[20293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474827, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.109.3@KRBTEST.COM for oryx/127.13.109.60@KRBTEST.COM
Aug 11 19:00:27 dist-test-slave-0kls krb5kdc[20293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474827, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.109.3@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM
I20260811 19:00:27.667472 21118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:27.648521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.3:54995 (local address 127.13.109.61:43661)
0811 19:00:27.648673 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:27.648678 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:27.649596 (+ 918us) server_negotiation.cc:407] Connection header received
0811 19:00:27.650870 (+ 1274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:27.650874 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:27.650935 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:27.651023 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:27.652924 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:27.653565 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:27.656246 (+ 2681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:27.656589 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:27.656665 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:27.663034 (+ 6369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:27.663039 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:27.663041 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:27.663073 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:27.664770 (+ 1697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:27.665499 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:27.665503 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:27.665504 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:27.665591 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:27.666060 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:27.666064 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:27.666066 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:27.666306 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:27.666399 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:27.667165 (+ 766us) server_negotiation.cc:299] Negotiation successful
0811 19:00:27.667292 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 19:00:27.668125 21386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:27.650166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.3:52291 (local address 127.13.109.60:34827)
0811 19:00:27.650321 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:27.650325 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:27.650397 (+ 72us) server_negotiation.cc:407] Connection header received
0811 19:00:27.650886 (+ 489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:27.650890 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:27.650950 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:27.651003 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:27.654325 (+ 3322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:27.655053 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:27.656360 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:27.656597 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:27.656707 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:27.663005 (+ 6298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:27.663011 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:27.663015 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:27.663067 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:27.665927 (+ 2860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:27.666445 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:27.666448 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:27.666450 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:27.666545 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:27.666830 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:27.666833 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:27.666835 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:27.667036 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:27.667129 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:27.667849 (+ 720us) server_negotiation.cc:299] Negotiation successful
0811 19:00:27.667966 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 19:00:27.668606 21692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:27.654137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52230 (local address 127.13.109.61:43661)
0811 19:00:27.657293 (+ 3156us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:27.657299 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:27.657314 (+ 15us) server_negotiation.cc:407] Connection header received
0811 19:00:27.657353 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:27.657357 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:27.657446 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:27.657551 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:27.659221 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:27.660082 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:27.661592 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:27.661972 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:27.662678 (+ 706us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:27.664420 (+ 1742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:27.664424 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:27.664427 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:27.664456 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:27.666326 (+ 1870us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:27.666846 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:27.666849 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:27.666851 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:27.666943 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:27.667259 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:27.667264 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:27.667267 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:27.667490 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:27.667631 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:27.668294 (+ 663us) server_negotiation.cc:299] Negotiation successful
0811 19:00:27.668425 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3072,"thread_start_us":90,"threads_started":1}
I20260811 19:00:27.669234 21682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:27.648959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.61:43661 (local address 127.13.109.3:54995)
0811 19:00:27.649433 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:27.649489 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:27.650654 (+ 1165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:27.651196 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:27.651203 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:27.651853 (+ 650us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:27.652735 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:27.652757 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:27.653746 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:27.653750 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:27.656122 (+ 2372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:27.656132 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:27.656832 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:27.656836 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:27.656910 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:27.657858 (+ 948us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:27.657872 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:27.662874 (+ 5002us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:00:27.664931 (+ 2057us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:27.664938 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:27.664958 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:27.665371 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:27.665715 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:27.665717 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:27.665719 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:27.665940 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:27.666476 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:00:27.666484 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:00:27.666854 (+ 370us) client_negotiation.cc:769] Sending connection context
0811 19:00:27.667702 (+ 848us) client_negotiation.cc:240] Negotiation successful
0811 19:00:27.667979 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"mutex_wait_us":1534,"spinlock_wait_cycles":48640,"thread_start_us":121,"threads_started":1}
I20260811 19:00:27.669332 20386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:27.648590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.3:55729 (local address 127.13.109.62:42815)
0811 19:00:27.648723 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:27.648728 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:27.649610 (+ 882us) server_negotiation.cc:407] Connection header received
0811 19:00:27.650887 (+ 1277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:27.650890 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:27.650947 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:27.651003 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:27.652944 (+ 1941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:27.653697 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:27.656047 (+ 2350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:27.656365 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:27.656434 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:27.661882 (+ 5448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:27.661886 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:27.661888 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:27.661917 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:27.665947 (+ 4030us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:27.667424 (+ 1477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:27.667427 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:27.667429 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:27.667514 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:27.667829 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:27.667832 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:27.667834 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:27.668067 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:27.668166 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:27.669058 (+ 892us) server_negotiation.cc:299] Negotiation successful
0811 19:00:27.669181 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 19:00:27.669579 21683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:27.648942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:42815 (local address 127.13.109.3:55729)
0811 19:00:27.649433 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:27.649489 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:27.650657 (+ 1168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:27.651197 (+ 540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:27.651204 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:27.651853 (+ 649us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:27.652735 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:27.652757 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:27.653832 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:27.653835 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:27.655295 (+ 1460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:27.655307 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:27.656654 (+ 1347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:27.656658 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:27.656860 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:27.657710 (+ 850us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:27.657727 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:27.661676 (+ 3949us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:00:27.666099 (+ 4423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:27.666103 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:27.666107 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:27.667308 (+ 1201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:27.667621 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:27.667624 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:27.667626 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:27.667719 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:27.668221 (+ 502us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:00:27.668224 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:00:27.668316 (+ 92us) client_negotiation.cc:769] Sending connection context
0811 19:00:27.668425 (+ 109us) client_negotiation.cc:240] Negotiation successful
0811 19:00:27.668574 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"mutex_wait_us":183,"spinlock_wait_cycles":53120,"thread_start_us":139,"threads_started":1}
I20260811 19:00:27.670192 21684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:27.650021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.60:34827 (local address 127.13.109.3:52291)
0811 19:00:27.650292 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:27.650313 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:27.650658 (+ 345us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:27.651197 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:27.651204 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:27.651924 (+ 720us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:27.654069 (+ 2145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:27.654082 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:27.655226 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:27.655230 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:27.656223 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:27.656236 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:27.656720 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:27.656724 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:27.656863 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:27.657701 (+ 838us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:27.657731 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:27.662507 (+ 4776us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:00:27.666051 (+ 3544us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:27.666054 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:27.666056 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:27.666334 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:27.666646 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:27.666648 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:27.666650 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:27.666731 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:27.667171 (+ 440us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:00:27.667176 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:00:27.667260 (+ 84us) client_negotiation.cc:769] Sending connection context
0811 19:00:27.667389 (+ 129us) client_negotiation.cc:240] Negotiation successful
0811 19:00:27.668838 (+ 1449us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"mutex_wait_us":247,"spinlock_wait_cycles":11264,"thread_start_us":81,"threads_started":1}
I20260811 19:00:27.670642 21679 heartbeater.cc:347] Connected to a master server at 127.13.109.62:42815
I20260811 19:00:27.670903 21679 heartbeater.cc:464] Registering TS with master...
I20260811 19:00:27.671229 21677 heartbeater.cc:347] Connected to a master server at 127.13.109.60:34827
I20260811 19:00:27.671322 21677 heartbeater.cc:464] Registering TS with master...
I20260811 19:00:27.671546 21677 heartbeater.cc:511] Master 127.13.109.60:34827 requested a full tablet report, sending...
I20260811 19:00:27.671628 21680 heartbeater.cc:347] Connected to a master server at 127.13.109.61:43661
I20260811 19:00:27.671707 21680 heartbeater.cc:464] Registering TS with master...
I20260811 19:00:27.671878 21680 heartbeater.cc:511] Master 127.13.109.61:43661 requested a full tablet report, sending...
I20260811 19:00:27.671556 21679 heartbeater.cc:511] Master 127.13.109.62:42815 requested a full tablet report, sending...
I20260811 19:00:27.672566 21695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:27.654817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46558 (local address 127.13.109.60:34827)
0811 19:00:27.655048 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:27.655052 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:27.657274 (+ 2222us) server_negotiation.cc:407] Connection header received
0811 19:00:27.657382 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:27.657384 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:27.657434 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:27.657521 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:27.659235 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:27.660001 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:27.661301 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:27.661659 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:27.661742 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:27.663687 (+ 1945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:27.663690 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:27.663695 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:27.663729 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:27.667030 (+ 3301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:27.670909 (+ 3879us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:27.670915 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:27.670920 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:27.671016 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:27.671503 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:27.671507 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:27.671509 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:27.671755 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:27.671879 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:27.672254 (+ 375us) server_negotiation.cc:299] Negotiation successful
0811 19:00:27.672402 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":83,"threads_started":1}
I20260811 19:00:27.672818 20983 ts_manager.cc:194] Registered new tserver with Master: 20f4f9eaa6844d8b9466235cf97ec8ce (127.13.109.3:43545)
I20260811 19:00:27.673064 20343 ts_manager.cc:194] Re-registered known tserver with Master: 20f4f9eaa6844d8b9466235cf97ec8ce (127.13.109.3:43545)
I20260811 19:00:27.673290 21182 ts_manager.cc:194] Registered new tserver with Master: 20f4f9eaa6844d8b9466235cf97ec8ce (127.13.109.3:43545)
I20260811 19:00:27.673899 20343 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.109.3@KRBTEST.COM'} at 127.13.109.3:55729
I20260811 19:00:27.675140 21691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:27.653754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50220 (local address 127.13.109.62:42815)
0811 19:00:27.655366 (+ 1612us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:27.655372 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:27.655385 (+ 13us) server_negotiation.cc:407] Connection header received
0811 19:00:27.655423 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:27.655426 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:27.655472 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:27.655605 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:27.657414 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:27.658221 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:27.659449 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:27.659814 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:27.660313 (+ 499us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:27.661644 (+ 1331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:27.661648 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:27.661651 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:27.661683 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:27.672732 (+ 11049us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:27.673663 (+ 931us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:27.673666 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:27.673668 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:27.673755 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:27.674150 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:27.674155 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:27.674157 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:27.674383 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:27.674510 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:27.674846 (+ 336us) server_negotiation.cc:299] Negotiation successful
0811 19:00:27.674974 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1522,"thread_start_us":92,"threads_started":1}
I20260811 19:00:27.677770 20343 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50220:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 19:00:27.681177 20343 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 19:00:27.684037 21533 sys_catalog.cc:455] T 00000000000000000000000000000000 P 449f60a2c460491d97c80e5cd420ef6e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "449f60a2c460491d97c80e5cd420ef6e" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "449f60a2c460491d97c80e5cd420ef6e" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 42815 } } peers { permanent_uuid: "26408052dd7842e1a0a036cce7c682c6" member_type: VOTER last_known_addr { host: "127.13.109.61" port: 43661 } attrs { promote: false } } peers { permanent_uuid: "adbbc0043cb14763a95172b87454443a" member_type: NON_VOTER last_known_addr { host: "127.13.109.60" port: 34827 } attrs { promote: true } } }
I20260811 19:00:27.684206 21533 sys_catalog.cc:458] T 00000000000000000000000000000000 P 449f60a2c460491d97c80e5cd420ef6e [sys.catalog]: This master's current role is: LEADER
I20260811 19:00:27.688230 21533 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 449f60a2c460491d97c80e5cd420ef6e: attempting to promote NON_VOTER adbbc0043cb14763a95172b87454443a to VOTER
I20260811 19:00:27.688601 21533 consensus_queue.cc:237] T 00000000000000000000000000000000 P 449f60a2c460491d97c80e5cd420ef6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "449f60a2c460491d97c80e5cd420ef6e" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 42815 } } peers { permanent_uuid: "26408052dd7842e1a0a036cce7c682c6" member_type: VOTER last_known_addr { host: "127.13.109.61" port: 43661 } attrs { promote: false } } peers { permanent_uuid: "adbbc0043cb14763a95172b87454443a" member_type: VOTER last_known_addr { host: "127.13.109.60" port: 34827 } attrs { promote: false } }
I20260811 19:00:27.690075 20993 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 26408052dd7842e1a0a036cce7c682c6 [term 1 FOLLOWER]: Refusing update from remote peer 449f60a2c460491d97c80e5cd420ef6e: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260811 19:00:27.690183 21193 raft_consensus.cc:1261] T 00000000000000000000000000000000 P adbbc0043cb14763a95172b87454443a [term 1 LEARNER]: Refusing update from remote peer 449f60a2c460491d97c80e5cd420ef6e: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260811 19:00:27.690452 21700 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 449f60a2c460491d97c80e5cd420ef6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "26408052dd7842e1a0a036cce7c682c6" member_type: VOTER last_known_addr { host: "127.13.109.61" port: 43661 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260811 19:00:27.690677 21698 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 449f60a2c460491d97c80e5cd420ef6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "adbbc0043cb14763a95172b87454443a" member_type: VOTER last_known_addr { host: "127.13.109.60" port: 34827 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260811 19:00:27.692386 21700 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 449f60a2c460491d97c80e5cd420ef6e [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, adbbc0043cb14763a95172b87454443a (127.13.109.60) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "449f60a2c460491d97c80e5cd420ef6e" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 42815 } } peers { permanent_uuid: "26408052dd7842e1a0a036cce7c682c6" member_type: VOTER last_known_addr { host: "127.13.109.61" port: 43661 } attrs { promote: false } } peers { permanent_uuid: "adbbc0043cb14763a95172b87454443a" member_type: VOTER last_known_addr { host: "127.13.109.60" port: 34827 } attrs { promote: false } } }
I20260811 19:00:27.693625 21700 sys_catalog.cc:455] T 00000000000000000000000000000000 P 449f60a2c460491d97c80e5cd420ef6e [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "449f60a2c460491d97c80e5cd420ef6e" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "449f60a2c460491d97c80e5cd420ef6e" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 42815 } } peers { permanent_uuid: "26408052dd7842e1a0a036cce7c682c6" member_type: VOTER last_known_addr { host: "127.13.109.61" port: 43661 } attrs { promote: false } } peers { permanent_uuid: "adbbc0043cb14763a95172b87454443a" member_type: VOTER last_known_addr { host: "127.13.109.60" port: 34827 } attrs { promote: false } } }
I20260811 19:00:27.693751 21700 sys_catalog.cc:458] T 00000000000000000000000000000000 P 449f60a2c460491d97c80e5cd420ef6e [sys.catalog]: This master's current role is: LEADER
I20260811 19:00:27.693717 21533 sys_catalog.cc:455] T 00000000000000000000000000000000 P 449f60a2c460491d97c80e5cd420ef6e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "449f60a2c460491d97c80e5cd420ef6e" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "449f60a2c460491d97c80e5cd420ef6e" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 42815 } } peers { permanent_uuid: "26408052dd7842e1a0a036cce7c682c6" member_type: VOTER last_known_addr { host: "127.13.109.61" port: 43661 } attrs { promote: false } } peers { permanent_uuid: "adbbc0043cb14763a95172b87454443a" member_type: VOTER last_known_addr { host: "127.13.109.60" port: 34827 } attrs { promote: false } } }
I20260811 19:00:27.693776 21699 sys_catalog.cc:455] T 00000000000000000000000000000000 P 449f60a2c460491d97c80e5cd420ef6e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "449f60a2c460491d97c80e5cd420ef6e" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "449f60a2c460491d97c80e5cd420ef6e" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 42815 } } peers { permanent_uuid: "26408052dd7842e1a0a036cce7c682c6" member_type: VOTER last_known_addr { host: "127.13.109.61" port: 43661 } attrs { promote: false } } peers { permanent_uuid: "adbbc0043cb14763a95172b87454443a" member_type: VOTER last_known_addr { host: "127.13.109.60" port: 34827 } attrs { promote: false } } }
I20260811 19:00:27.693866 21533 sys_catalog.cc:458] T 00000000000000000000000000000000 P 449f60a2c460491d97c80e5cd420ef6e [sys.catalog]: This master's current role is: LEADER
I20260811 19:00:27.693913 21699 sys_catalog.cc:458] T 00000000000000000000000000000000 P 449f60a2c460491d97c80e5cd420ef6e [sys.catalog]: This master's current role is: LEADER
I20260811 19:00:27.694442 20993 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 26408052dd7842e1a0a036cce7c682c6 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, adbbc0043cb14763a95172b87454443a (127.13.109.60) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "449f60a2c460491d97c80e5cd420ef6e" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 42815 } } peers { permanent_uuid: "26408052dd7842e1a0a036cce7c682c6" member_type: VOTER last_known_addr { host: "127.13.109.61" port: 43661 } attrs { promote: false } } peers { permanent_uuid: "adbbc0043cb14763a95172b87454443a" member_type: VOTER last_known_addr { host: "127.13.109.60" port: 34827 } attrs { promote: false } } }
I20260811 19:00:27.694432 21193 raft_consensus.cc:2932] T 00000000000000000000000000000000 P adbbc0043cb14763a95172b87454443a [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, adbbc0043cb14763a95172b87454443a (127.13.109.60) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "449f60a2c460491d97c80e5cd420ef6e" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 42815 } } peers { permanent_uuid: "26408052dd7842e1a0a036cce7c682c6" member_type: VOTER last_known_addr { host: "127.13.109.61" port: 43661 } attrs { promote: false } } peers { permanent_uuid: "adbbc0043cb14763a95172b87454443a" member_type: VOTER last_known_addr { host: "127.13.109.60" port: 34827 } attrs { promote: false } } }
I20260811 19:00:27.696254 21229 sys_catalog.cc:455] T 00000000000000000000000000000000 P adbbc0043cb14763a95172b87454443a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "449f60a2c460491d97c80e5cd420ef6e" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "449f60a2c460491d97c80e5cd420ef6e" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 42815 } } peers { permanent_uuid: "26408052dd7842e1a0a036cce7c682c6" member_type: VOTER last_known_addr { host: "127.13.109.61" port: 43661 } attrs { promote: false } } peers { permanent_uuid: "adbbc0043cb14763a95172b87454443a" member_type: VOTER last_known_addr { host: "127.13.109.60" port: 34827 } attrs { promote: false } } }
I20260811 19:00:27.696375 21229 sys_catalog.cc:458] T 00000000000000000000000000000000 P adbbc0043cb14763a95172b87454443a [sys.catalog]: This master's current role is: FOLLOWER
I20260811 19:00:27.700050 21711 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26408052dd7842e1a0a036cce7c682c6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "449f60a2c460491d97c80e5cd420ef6e" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "449f60a2c460491d97c80e5cd420ef6e" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 42815 } } peers { permanent_uuid: "26408052dd7842e1a0a036cce7c682c6" member_type: VOTER last_known_addr { host: "127.13.109.61" port: 43661 } attrs { promote: false } } peers { permanent_uuid: "adbbc0043cb14763a95172b87454443a" member_type: VOTER last_known_addr { host: "127.13.109.60" port: 34827 } attrs { promote: false } } }
I20260811 19:00:27.700223 21711 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26408052dd7842e1a0a036cce7c682c6 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 19:00:27.705117 21550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:27.698859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.2:32839 (local address 127.0.0.1:46588)
0811 19:00:27.699001 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:27.699018 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:27.699104 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:27.701075 (+ 1971us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:27.701078 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:27.701097 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:00:27.701176 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:27.701181 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:27.702529 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:27.702533 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:27.703712 (+ 1179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:27.703722 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:27.704627 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:27.704630 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:27.704752 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:27.704800 (+ 48us) client_negotiation.cc:769] Sending connection context
0811 19:00:27.704870 (+ 70us) client_negotiation.cc:240] Negotiation successful
0811 19:00:27.704950 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52,"mutex_wait_us":50}
I20260811 19:00:27.705132 21713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:27.699248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46588 (local address 127.13.109.2:32839)
0811 19:00:27.700733 (+ 1485us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:27.700742 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:27.700759 (+ 17us) server_negotiation.cc:407] Connection header received
0811 19:00:27.700819 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:27.700823 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:27.700948 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:27.701060 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:00:27.701310 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:27.702420 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:27.703834 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:27.704496 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:27.704616 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:27.704761 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:27.704899 (+ 138us) server_negotiation.cc:299] Negotiation successful
0811 19:00:27.704995 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1399,"thread_start_us":120,"threads_started":1}
I20260811 19:00:27.705272 21714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:27.699615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39284 (local address 127.13.109.1:38457)
0811 19:00:27.699961 (+ 346us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:27.699980 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:27.700006 (+ 26us) server_negotiation.cc:407] Connection header received
0811 19:00:27.700106 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:27.700112 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:27.700407 (+ 295us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:27.700581 (+ 174us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:00:27.700822 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:27.701960 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:27.703449 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:27.704283 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:27.704387 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:27.704556 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:27.705028 (+ 472us) server_negotiation.cc:299] Negotiation successful
0811 19:00:27.705127 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":236,"thread_start_us":134,"threads_started":1}
I20260811 19:00:27.705488 21712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:27.699128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.1:38457 (local address 127.0.0.1:39284)
0811 19:00:27.699428 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:27.699443 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:27.699538 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:27.700550 (+ 1012us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:27.700553 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:27.700573 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:00:27.700701 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:27.700708 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:27.702074 (+ 1366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:27.702077 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:27.703321 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:27.703329 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:27.704631 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:27.704636 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:27.704757 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:27.704813 (+ 56us) client_negotiation.cc:769] Sending connection context
0811 19:00:27.704882 (+ 69us) client_negotiation.cc:240] Negotiation successful
0811 19:00:27.704949 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"mutex_wait_us":64,"thread_start_us":115,"threads_started":1}
I20260811 19:00:27.705953 21716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:27.699657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.3:43545 (local address 127.0.0.1:35518)
0811 19:00:27.700440 (+ 783us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:27.700452 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:27.700527 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:27.701665 (+ 1138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:27.701668 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:27.701687 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:00:27.701807 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:27.701817 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:27.703250 (+ 1433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:27.703254 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:27.704406 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:27.704417 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:27.705499 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:27.705502 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:27.705620 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:27.705656 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 19:00:27.705708 (+ 52us) client_negotiation.cc:240] Negotiation successful
0811 19:00:27.705817 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":731,"thread_start_us":107,"threads_started":1}
I20260811 19:00:27.706506 21717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:27.700053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35518 (local address 127.13.109.3:43545)
0811 19:00:27.701311 (+ 1258us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:27.701319 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:27.701338 (+ 19us) server_negotiation.cc:407] Connection header received
0811 19:00:27.701396 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:27.701400 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:27.701524 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:27.701680 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:00:27.701908 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:27.703126 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:27.704555 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:27.705359 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:27.706152 (+ 793us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:27.706278 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:27.706318 (+ 40us) server_negotiation.cc:299] Negotiation successful
0811 19:00:27.706382 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1167,"thread_start_us":95,"threads_started":1}
I20260811 19:00:27.708743 21603 tablet_service.cc:1467] Processing CreateTablet for tablet e01ec8316cd84bf3a913028676d21623 (DEFAULT_TABLE table=test-table [id=e679d8004bbf4dd8b7e4d595b660e191]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:00:27.708792 21455 tablet_service.cc:1467] Processing CreateTablet for tablet e01ec8316cd84bf3a913028676d21623 (DEFAULT_TABLE table=test-table [id=e679d8004bbf4dd8b7e4d595b660e191]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:00:27.708792 21305 tablet_service.cc:1467] Processing CreateTablet for tablet e01ec8316cd84bf3a913028676d21623 (DEFAULT_TABLE table=test-table [id=e679d8004bbf4dd8b7e4d595b660e191]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:00:27.709647 21603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e01ec8316cd84bf3a913028676d21623. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:00:27.709776 21305 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e01ec8316cd84bf3a913028676d21623. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:00:27.709776 21455 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e01ec8316cd84bf3a913028676d21623. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:00:27.716688 21718 tablet_bootstrap.cc:492] T e01ec8316cd84bf3a913028676d21623 P 20f4f9eaa6844d8b9466235cf97ec8ce: Bootstrap starting.
I20260811 19:00:27.718357 21719 tablet_bootstrap.cc:492] T e01ec8316cd84bf3a913028676d21623 P adf5bc4ddc704fec824189012815251a: Bootstrap starting.
I20260811 19:00:27.719377 21718 tablet_bootstrap.cc:654] T e01ec8316cd84bf3a913028676d21623 P 20f4f9eaa6844d8b9466235cf97ec8ce: Neither blocks nor log segments found. Creating new log.
I20260811 19:00:27.720078 21719 tablet_bootstrap.cc:654] T e01ec8316cd84bf3a913028676d21623 P adf5bc4ddc704fec824189012815251a: Neither blocks nor log segments found. Creating new log.
I20260811 19:00:27.720211 21718 log.cc:824] T e01ec8316cd84bf3a913028676d21623 P 20f4f9eaa6844d8b9466235cf97ec8ce: Log is configured to *not* fsync() on all Append() calls
I20260811 19:00:27.720896 21719 log.cc:824] T e01ec8316cd84bf3a913028676d21623 P adf5bc4ddc704fec824189012815251a: Log is configured to *not* fsync() on all Append() calls
I20260811 19:00:27.721575 21720 tablet_bootstrap.cc:492] T e01ec8316cd84bf3a913028676d21623 P 2c9427fe9258493f9465751fef6bb43a: Bootstrap starting.
I20260811 19:00:27.722409 21718 tablet_bootstrap.cc:492] T e01ec8316cd84bf3a913028676d21623 P 20f4f9eaa6844d8b9466235cf97ec8ce: No bootstrap required, opened a new log
I20260811 19:00:27.722602 21718 ts_tablet_manager.cc:1397] T e01ec8316cd84bf3a913028676d21623 P 20f4f9eaa6844d8b9466235cf97ec8ce: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 19:00:27.723551 21720 tablet_bootstrap.cc:654] T e01ec8316cd84bf3a913028676d21623 P 2c9427fe9258493f9465751fef6bb43a: Neither blocks nor log segments found. Creating new log.
I20260811 19:00:27.724285 21720 log.cc:824] T e01ec8316cd84bf3a913028676d21623 P 2c9427fe9258493f9465751fef6bb43a: Log is configured to *not* fsync() on all Append() calls
I20260811 19:00:27.726269 21720 tablet_bootstrap.cc:492] T e01ec8316cd84bf3a913028676d21623 P 2c9427fe9258493f9465751fef6bb43a: No bootstrap required, opened a new log
I20260811 19:00:27.726477 21720 ts_tablet_manager.cc:1397] T e01ec8316cd84bf3a913028676d21623 P 2c9427fe9258493f9465751fef6bb43a: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 19:00:27.727559 21718 raft_consensus.cc:348] T e01ec8316cd84bf3a913028676d21623 P 20f4f9eaa6844d8b9466235cf97ec8ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adf5bc4ddc704fec824189012815251a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 32839 } } peers { permanent_uuid: "2c9427fe9258493f9465751fef6bb43a" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 38457 } } peers { permanent_uuid: "20f4f9eaa6844d8b9466235cf97ec8ce" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43545 } }
I20260811 19:00:27.727846 21718 raft_consensus.cc:374] T e01ec8316cd84bf3a913028676d21623 P 20f4f9eaa6844d8b9466235cf97ec8ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:00:27.727890 21718 raft_consensus.cc:729] T e01ec8316cd84bf3a913028676d21623 P 20f4f9eaa6844d8b9466235cf97ec8ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20f4f9eaa6844d8b9466235cf97ec8ce, State: Initialized, Role: FOLLOWER
I20260811 19:00:27.728343 21718 consensus_queue.cc:260] T e01ec8316cd84bf3a913028676d21623 P 20f4f9eaa6844d8b9466235cf97ec8ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adf5bc4ddc704fec824189012815251a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 32839 } } peers { permanent_uuid: "2c9427fe9258493f9465751fef6bb43a" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 38457 } } peers { permanent_uuid: "20f4f9eaa6844d8b9466235cf97ec8ce" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43545 } }
I20260811 19:00:27.729380 21718 ts_tablet_manager.cc:1428] T e01ec8316cd84bf3a913028676d21623 P 20f4f9eaa6844d8b9466235cf97ec8ce: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260811 19:00:27.730988 21720 raft_consensus.cc:348] T e01ec8316cd84bf3a913028676d21623 P 2c9427fe9258493f9465751fef6bb43a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adf5bc4ddc704fec824189012815251a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 32839 } } peers { permanent_uuid: "2c9427fe9258493f9465751fef6bb43a" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 38457 } } peers { permanent_uuid: "20f4f9eaa6844d8b9466235cf97ec8ce" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43545 } }
I20260811 19:00:27.731287 21720 raft_consensus.cc:374] T e01ec8316cd84bf3a913028676d21623 P 2c9427fe9258493f9465751fef6bb43a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:00:27.731361 21720 raft_consensus.cc:729] T e01ec8316cd84bf3a913028676d21623 P 2c9427fe9258493f9465751fef6bb43a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c9427fe9258493f9465751fef6bb43a, State: Initialized, Role: FOLLOWER
I20260811 19:00:27.731673 21719 tablet_bootstrap.cc:492] T e01ec8316cd84bf3a913028676d21623 P adf5bc4ddc704fec824189012815251a: No bootstrap required, opened a new log
I20260811 19:00:27.731931 21719 ts_tablet_manager.cc:1397] T e01ec8316cd84bf3a913028676d21623 P adf5bc4ddc704fec824189012815251a: Time spent bootstrapping tablet: real 0.014s user 0.005s sys 0.000s
I20260811 19:00:27.731878 21720 consensus_queue.cc:260] T e01ec8316cd84bf3a913028676d21623 P 2c9427fe9258493f9465751fef6bb43a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adf5bc4ddc704fec824189012815251a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 32839 } } peers { permanent_uuid: "2c9427fe9258493f9465751fef6bb43a" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 38457 } } peers { permanent_uuid: "20f4f9eaa6844d8b9466235cf97ec8ce" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43545 } }
I20260811 19:00:27.732740 21679 heartbeater.cc:503] Master 127.13.109.62:42815 was elected leader, sending a full tablet report...
I20260811 19:00:27.732934 21720 ts_tablet_manager.cc:1428] T e01ec8316cd84bf3a913028676d21623 P 2c9427fe9258493f9465751fef6bb43a: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
I20260811 19:00:27.734490 21377 heartbeater.cc:503] Master 127.13.109.62:42815 was elected leader, sending a full tablet report...
I20260811 19:00:27.737547 21719 raft_consensus.cc:348] T e01ec8316cd84bf3a913028676d21623 P adf5bc4ddc704fec824189012815251a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adf5bc4ddc704fec824189012815251a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 32839 } } peers { permanent_uuid: "2c9427fe9258493f9465751fef6bb43a" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 38457 } } peers { permanent_uuid: "20f4f9eaa6844d8b9466235cf97ec8ce" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43545 } }
I20260811 19:00:27.737964 21719 raft_consensus.cc:374] T e01ec8316cd84bf3a913028676d21623 P adf5bc4ddc704fec824189012815251a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:00:27.738057 21719 raft_consensus.cc:729] T e01ec8316cd84bf3a913028676d21623 P adf5bc4ddc704fec824189012815251a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: adf5bc4ddc704fec824189012815251a, State: Initialized, Role: FOLLOWER
I20260811 19:00:27.738559 21719 consensus_queue.cc:260] T e01ec8316cd84bf3a913028676d21623 P adf5bc4ddc704fec824189012815251a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adf5bc4ddc704fec824189012815251a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 32839 } } peers { permanent_uuid: "2c9427fe9258493f9465751fef6bb43a" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 38457 } } peers { permanent_uuid: "20f4f9eaa6844d8b9466235cf97ec8ce" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43545 } }
I20260811 19:00:27.739688 21719 ts_tablet_manager.cc:1428] T e01ec8316cd84bf3a913028676d21623 P adf5bc4ddc704fec824189012815251a: Time spent starting tablet: real 0.008s user 0.000s sys 0.006s
I20260811 19:00:27.740118 21526 heartbeater.cc:503] Master 127.13.109.62:42815 was elected leader, sending a full tablet report...
I20260811 19:00:27.842097 21726 raft_consensus.cc:482] T e01ec8316cd84bf3a913028676d21623 P adf5bc4ddc704fec824189012815251a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:00:27.842367 21726 raft_consensus.cc:504] T e01ec8316cd84bf3a913028676d21623 P adf5bc4ddc704fec824189012815251a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adf5bc4ddc704fec824189012815251a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 32839 } } peers { permanent_uuid: "2c9427fe9258493f9465751fef6bb43a" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 38457 } } peers { permanent_uuid: "20f4f9eaa6844d8b9466235cf97ec8ce" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43545 } }
I20260811 19:00:27.843686 21726 leader_election.cc:288] T e01ec8316cd84bf3a913028676d21623 P adf5bc4ddc704fec824189012815251a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c9427fe9258493f9465751fef6bb43a (127.13.109.1:38457), 20f4f9eaa6844d8b9466235cf97ec8ce (127.13.109.3:43545)
I20260811 19:00:27.847715 21717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:27.844134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.2:44517 (local address 127.13.109.3:43545)
0811 19:00:27.844265 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:27.844270 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:27.844380 (+ 110us) server_negotiation.cc:407] Connection header received
0811 19:00:27.844475 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:27.844478 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:27.844518 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:27.844585 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:00:27.844969 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:27.845563 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:27.846727 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:27.847128 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:27.847249 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:27.847308 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:27.847489 (+ 181us) server_negotiation.cc:299] Negotiation successful
0811 19:00:27.847576 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 19:00:27.847723 21530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:27.843994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.1:38457 (local address 127.13.109.2:58071)
0811 19:00:27.844215 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:27.844233 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:27.844335 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:27.844679 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:27.844683 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:27.844704 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:00:27.844838 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:27.844843 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:27.845588 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:27.845594 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:27.846613 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:27.846623 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:27.847288 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:27.847292 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:27.847396 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:27.847412 (+ 16us) client_negotiation.cc:769] Sending connection context
0811 19:00:27.847510 (+ 98us) client_negotiation.cc:240] Negotiation successful
0811 19:00:27.847576 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"mutex_wait_us":67}
I20260811 19:00:27.847735 21714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:27.844288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.2:58071 (local address 127.13.109.1:38457)
0811 19:00:27.844425 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:27.844432 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:27.844448 (+ 16us) server_negotiation.cc:407] Connection header received
0811 19:00:27.844510 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:27.844513 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:27.844551 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:27.844648 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:00:27.844953 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:27.845483 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:27.846762 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:27.847149 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:27.847281 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:27.847336 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:27.847541 (+ 205us) server_negotiation.cc:299] Negotiation successful
0811 19:00:27.847624 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 19:00:27.847735 21531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:27.843995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.3:43545 (local address 127.13.109.2:44517)
0811 19:00:27.844271 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:27.844286 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:27.844382 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:27.844629 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:27.844635 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:27.844653 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:00:27.844839 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:27.844846 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:27.845698 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:27.845701 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:27.846602 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:27.846611 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:27.847249 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:27.847252 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:27.847345 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:27.847375 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 19:00:27.847444 (+ 69us) client_negotiation.cc:240] Negotiation successful
0811 19:00:27.847543 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"mutex_wait_us":130}
I20260811 19:00:27.848574 21627 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e01ec8316cd84bf3a913028676d21623" candidate_uuid: "adf5bc4ddc704fec824189012815251a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20f4f9eaa6844d8b9466235cf97ec8ce" is_pre_election: true
I20260811 19:00:27.848635 21327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e01ec8316cd84bf3a913028676d21623" candidate_uuid: "adf5bc4ddc704fec824189012815251a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9427fe9258493f9465751fef6bb43a" is_pre_election: true
I20260811 19:00:27.848927 21627 raft_consensus.cc:2454] T e01ec8316cd84bf3a913028676d21623 P 20f4f9eaa6844d8b9466235cf97ec8ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate adf5bc4ddc704fec824189012815251a in term 0.
I20260811 19:00:27.848948 21327 raft_consensus.cc:2454] T e01ec8316cd84bf3a913028676d21623 P 2c9427fe9258493f9465751fef6bb43a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate adf5bc4ddc704fec824189012815251a in term 0.
I20260811 19:00:27.849536 21407 leader_election.cc:302] T e01ec8316cd84bf3a913028676d21623 P adf5bc4ddc704fec824189012815251a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 20f4f9eaa6844d8b9466235cf97ec8ce, 2c9427fe9258493f9465751fef6bb43a, adf5bc4ddc704fec824189012815251a; no voters:
I20260811 19:00:27.849915 21726 raft_consensus.cc:2781] T e01ec8316cd84bf3a913028676d21623 P adf5bc4ddc704fec824189012815251a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 19:00:27.850021 21726 raft_consensus.cc:482] T e01ec8316cd84bf3a913028676d21623 P adf5bc4ddc704fec824189012815251a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 19:00:27.850107 21726 raft_consensus.cc:3037] T e01ec8316cd84bf3a913028676d21623 P adf5bc4ddc704fec824189012815251a [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:00:27.851341 21726 raft_consensus.cc:504] T e01ec8316cd84bf3a913028676d21623 P adf5bc4ddc704fec824189012815251a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adf5bc4ddc704fec824189012815251a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 32839 } } peers { permanent_uuid: "2c9427fe9258493f9465751fef6bb43a" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 38457 } } peers { permanent_uuid: "20f4f9eaa6844d8b9466235cf97ec8ce" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43545 } }
I20260811 19:00:27.851765 21726 leader_election.cc:288] T e01ec8316cd84bf3a913028676d21623 P adf5bc4ddc704fec824189012815251a [CANDIDATE]: Term 1 election: Requested vote from peers 2c9427fe9258493f9465751fef6bb43a (127.13.109.1:38457), 20f4f9eaa6844d8b9466235cf97ec8ce (127.13.109.3:43545)
I20260811 19:00:27.852218 21327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e01ec8316cd84bf3a913028676d21623" candidate_uuid: "adf5bc4ddc704fec824189012815251a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9427fe9258493f9465751fef6bb43a"
I20260811 19:00:27.852385 21327 raft_consensus.cc:3037] T e01ec8316cd84bf3a913028676d21623 P 2c9427fe9258493f9465751fef6bb43a [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:00:27.853487 21327 raft_consensus.cc:2454] T e01ec8316cd84bf3a913028676d21623 P 2c9427fe9258493f9465751fef6bb43a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate adf5bc4ddc704fec824189012815251a in term 1.
I20260811 19:00:27.853842 21627 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e01ec8316cd84bf3a913028676d21623" candidate_uuid: "adf5bc4ddc704fec824189012815251a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20f4f9eaa6844d8b9466235cf97ec8ce"
I20260811 19:00:27.853911 21405 leader_election.cc:302] T e01ec8316cd84bf3a913028676d21623 P adf5bc4ddc704fec824189012815251a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9427fe9258493f9465751fef6bb43a, adf5bc4ddc704fec824189012815251a; no voters:
I20260811 19:00:27.853995 21627 raft_consensus.cc:3037] T e01ec8316cd84bf3a913028676d21623 P 20f4f9eaa6844d8b9466235cf97ec8ce [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:00:27.854120 21726 raft_consensus.cc:2781] T e01ec8316cd84bf3a913028676d21623 P adf5bc4ddc704fec824189012815251a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:00:27.854368 21726 raft_consensus.cc:686] T e01ec8316cd84bf3a913028676d21623 P adf5bc4ddc704fec824189012815251a [term 1 LEADER]: Becoming Leader. State: Replica: adf5bc4ddc704fec824189012815251a, State: Running, Role: LEADER
I20260811 19:00:27.854763 21726 consensus_queue.cc:237] T e01ec8316cd84bf3a913028676d21623 P adf5bc4ddc704fec824189012815251a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adf5bc4ddc704fec824189012815251a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 32839 } } peers { permanent_uuid: "2c9427fe9258493f9465751fef6bb43a" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 38457 } } peers { permanent_uuid: "20f4f9eaa6844d8b9466235cf97ec8ce" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43545 } }
I20260811 19:00:27.855453 21627 raft_consensus.cc:2454] T e01ec8316cd84bf3a913028676d21623 P 20f4f9eaa6844d8b9466235cf97ec8ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate adf5bc4ddc704fec824189012815251a in term 1.
I20260811 19:00:27.858505 20343 catalog_manager.cc:5495] T e01ec8316cd84bf3a913028676d21623 P adf5bc4ddc704fec824189012815251a reported cstate change: term changed from 0 to 1, leader changed from <none> to adf5bc4ddc704fec824189012815251a (127.13.109.2). New cstate: current_term: 1 leader_uuid: "adf5bc4ddc704fec824189012815251a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adf5bc4ddc704fec824189012815251a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 32839 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c9427fe9258493f9465751fef6bb43a" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 38457 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20f4f9eaa6844d8b9466235cf97ec8ce" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43545 } health_report { overall_health: UNKNOWN } } }
I20260811 19:00:27.896476 21713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:27.892156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46596 (local address 127.13.109.2:32839)
0811 19:00:27.892305 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:27.892309 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:27.892323 (+ 14us) server_negotiation.cc:407] Connection header received
0811 19:00:27.892528 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:27.892530 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:27.892623 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:27.892763 (+ 140us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:00:27.893108 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:27.893614 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:27.894681 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:27.894929 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:27.895081 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:27.895462 (+ 381us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:00:27.895974 (+ 512us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:00:27.896077 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:27.896224 (+ 147us) server_negotiation.cc:299] Negotiation successful
0811 19:00:27.896292 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
W20260811 19:00:27.898288 21681 tablet.cc:2369] T e01ec8316cd84bf3a913028676d21623 P 20f4f9eaa6844d8b9466235cf97ec8ce: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 19:00:27.903009 21327 raft_consensus.cc:1261] T e01ec8316cd84bf3a913028676d21623 P 2c9427fe9258493f9465751fef6bb43a [term 1 FOLLOWER]: Refusing update from remote peer adf5bc4ddc704fec824189012815251a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:00:27.903021 21627 raft_consensus.cc:1261] T e01ec8316cd84bf3a913028676d21623 P 20f4f9eaa6844d8b9466235cf97ec8ce [term 1 FOLLOWER]: Refusing update from remote peer adf5bc4ddc704fec824189012815251a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:00:27.903838 21726 consensus_queue.cc:1035] T e01ec8316cd84bf3a913028676d21623 P adf5bc4ddc704fec824189012815251a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9427fe9258493f9465751fef6bb43a" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 38457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:00:27.904107 21727 consensus_queue.cc:1035] T e01ec8316cd84bf3a913028676d21623 P adf5bc4ddc704fec824189012815251a [LEADER]: Connected to new peer: Peer: permanent_uuid: "20f4f9eaa6844d8b9466235cf97ec8ce" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:00:27.915217 21730 mvcc.cc:205] Tried to move back new op lower bound from 7317400895082315776 to 7317400894895910912. Current Snapshot: MvccSnapshot[applied={T|T < 7317400895082315776}]
I20260811 19:00:27.916101 21731 mvcc.cc:205] Tried to move back new op lower bound from 7317400895082315776 to 7317400894895910912. Current Snapshot: MvccSnapshot[applied={T|T < 7317400895082315776}]
I20260811 19:00:27.916730 21732 mvcc.cc:205] Tried to move back new op lower bound from 7317400895082315776 to 7317400894895910912. Current Snapshot: MvccSnapshot[applied={T|T < 7317400895082315776}]
I20260811 19:00:27.926147 20343 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50220:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:00:27.926352 20343 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50220:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:00:27.931541 20343 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 449f60a2c460491d97c80e5cd420ef6e: Sending DeleteTablet for 3 replicas of tablet e01ec8316cd84bf3a913028676d21623
I20260811 19:00:27.932417 21455 tablet_service.cc:1514] Processing DeleteTablet for tablet e01ec8316cd84bf3a913028676d21623 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:00:27 UTC) from {username='oryx', principal='oryx/127.13.109.62@KRBTEST.COM'} at 127.0.0.1:46588
I20260811 19:00:27.932828 21305 tablet_service.cc:1514] Processing DeleteTablet for tablet e01ec8316cd84bf3a913028676d21623 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:00:27 UTC) from {username='oryx', principal='oryx/127.13.109.62@KRBTEST.COM'} at 127.0.0.1:39284
I20260811 19:00:27.932840 21603 tablet_service.cc:1514] Processing DeleteTablet for tablet e01ec8316cd84bf3a913028676d21623 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:00:27 UTC) from {username='oryx', principal='oryx/127.13.109.62@KRBTEST.COM'} at 127.0.0.1:35518
I20260811 19:00:27.933380 21743 tablet_replica.cc:330] T e01ec8316cd84bf3a913028676d21623 P 20f4f9eaa6844d8b9466235cf97ec8ce: stopping tablet replica
I20260811 19:00:27.933396 21742 tablet_replica.cc:330] T e01ec8316cd84bf3a913028676d21623 P adf5bc4ddc704fec824189012815251a: stopping tablet replica
I20260811 19:00:27.933758 21743 raft_consensus.cc:2229] T e01ec8316cd84bf3a913028676d21623 P 20f4f9eaa6844d8b9466235cf97ec8ce [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:00:27.933764 21742 raft_consensus.cc:2229] T e01ec8316cd84bf3a913028676d21623 P adf5bc4ddc704fec824189012815251a [term 1 LEADER]: Raft consensus shutting down.
I20260811 19:00:27.934330 21742 raft_consensus.cc:2258] T e01ec8316cd84bf3a913028676d21623 P adf5bc4ddc704fec824189012815251a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:00:27.934386 21744 tablet_replica.cc:330] T e01ec8316cd84bf3a913028676d21623 P 2c9427fe9258493f9465751fef6bb43a: stopping tablet replica
I20260811 19:00:27.934698 21744 raft_consensus.cc:2229] T e01ec8316cd84bf3a913028676d21623 P 2c9427fe9258493f9465751fef6bb43a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:00:27.935005 21744 raft_consensus.cc:2258] T e01ec8316cd84bf3a913028676d21623 P 2c9427fe9258493f9465751fef6bb43a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:00:27.935230 21743 raft_consensus.cc:2258] T e01ec8316cd84bf3a913028676d21623 P 20f4f9eaa6844d8b9466235cf97ec8ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:00:27.936641 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 20308
I20260811 19:00:27.939138 21743 ts_tablet_manager.cc:1905] T e01ec8316cd84bf3a913028676d21623 P 20f4f9eaa6844d8b9466235cf97ec8ce: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:00:27.939492 21744 ts_tablet_manager.cc:1905] T e01ec8316cd84bf3a913028676d21623 P 2c9427fe9258493f9465751fef6bb43a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:00:27.942495 21744 ts_tablet_manager.cc:1918] T e01ec8316cd84bf3a913028676d21623 P 2c9427fe9258493f9465751fef6bb43a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:00:27.942605 21744 log.cc:1196] T e01ec8316cd84bf3a913028676d21623 P 2c9427fe9258493f9465751fef6bb43a: Deleting WAL directory at /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/ts-0/wal/wals/e01ec8316cd84bf3a913028676d21623
I20260811 19:00:27.942865 21743 ts_tablet_manager.cc:1918] T e01ec8316cd84bf3a913028676d21623 P 20f4f9eaa6844d8b9466235cf97ec8ce: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:00:27.942940 21744 ts_tablet_manager.cc:1939] T e01ec8316cd84bf3a913028676d21623 P 2c9427fe9258493f9465751fef6bb43a: Deleting consensus metadata
I20260811 19:00:27.942972 21743 log.cc:1196] T e01ec8316cd84bf3a913028676d21623 P 20f4f9eaa6844d8b9466235cf97ec8ce: Deleting WAL directory at /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/ts-2/wal/wals/e01ec8316cd84bf3a913028676d21623
I20260811 19:00:27.943311 21743 ts_tablet_manager.cc:1939] T e01ec8316cd84bf3a913028676d21623 P 20f4f9eaa6844d8b9466235cf97ec8ce: Deleting consensus metadata
W20260811 19:00:27.947654 21553 connection.cc:535] server connection from 127.0.0.1:35518 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 19:00:27.948061 21259 connection.cc:535] server connection from 127.0.0.1:39284 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 19:00:27.949301 21742 ts_tablet_manager.cc:1905] T e01ec8316cd84bf3a913028676d21623 P adf5bc4ddc704fec824189012815251a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:00:27.955889 21742 ts_tablet_manager.cc:1918] T e01ec8316cd84bf3a913028676d21623 P adf5bc4ddc704fec824189012815251a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:00:27.956033 21742 log.cc:1196] T e01ec8316cd84bf3a913028676d21623 P adf5bc4ddc704fec824189012815251a: Deleting WAL directory at /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/ts-1/wal/wals/e01ec8316cd84bf3a913028676d21623
I20260811 19:00:27.956434 21742 ts_tablet_manager.cc:1939] T e01ec8316cd84bf3a913028676d21623 P adf5bc4ddc704fec824189012815251a: Deleting consensus metadata
I20260811 19:00:27.956797 21692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:27.935010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52242 (local address 127.13.109.61:43661)
0811 19:00:27.935161 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:27.935165 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:27.944761 (+ 9596us) server_negotiation.cc:407] Connection header received
0811 19:00:27.944892 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:27.944895 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:27.944955 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:27.945050 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:27.946160 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:27.947001 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:27.948250 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:27.948708 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:27.948790 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:27.952689 (+ 3899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:27.952695 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:27.952698 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:27.952739 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:27.954678 (+ 1939us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:27.955271 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:27.955276 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:27.955278 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:27.955379 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:27.955716 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:27.955720 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:27.955722 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:27.955970 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:27.956098 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:27.956457 (+ 359us) server_negotiation.cc:299] Negotiation successful
0811 19:00:27.956625 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
W20260811 19:00:27.957849 21406 connection.cc:441] server connection from 127.0.0.1:46588 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46588 (request call id 1) could send its response
I20260811 19:00:27.969211 21692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:27.958176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52258 (local address 127.13.109.61:43661)
0811 19:00:27.958303 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:27.958309 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:27.959025 (+ 716us) server_negotiation.cc:407] Connection header received
0811 19:00:27.959151 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:27.959154 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:27.959199 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:27.959271 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:27.960051 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:27.960876 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:27.961731 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:27.962110 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:27.962218 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:27.965559 (+ 3341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:27.965564 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:27.965567 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:27.965598 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:27.967222 (+ 1624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:27.967767 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:27.967771 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:27.967773 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:27.967867 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:27.968170 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:27.968173 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:27.968175 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:27.968399 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:27.968501 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:27.968894 (+ 393us) server_negotiation.cc:299] Negotiation successful
0811 19:00:27.969029 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 19:00:27.969928 21695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:27.935817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46564 (local address 127.13.109.60:34827)
0811 19:00:27.935943 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:27.935948 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:27.958411 (+ 22463us) server_negotiation.cc:407] Connection header received
0811 19:00:27.958611 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:27.958615 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:27.958669 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:27.958721 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:27.959619 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:27.960528 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:27.961880 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:27.962231 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:27.962346 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:27.965118 (+ 2772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:27.965123 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:27.965126 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:27.965157 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:27.967742 (+ 2585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:27.968230 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:27.968236 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:27.968238 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:27.968326 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:27.968700 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:27.968705 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:27.968710 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:27.968956 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:27.969075 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:27.969485 (+ 410us) server_negotiation.cc:299] Negotiation successful
0811 19:00:27.969653 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 19:00:27.972110 21386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:27.958545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46580 (local address 127.13.109.60:34827)
0811 19:00:27.958662 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:27.958667 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:27.962670 (+ 4003us) server_negotiation.cc:407] Connection header received
0811 19:00:27.962816 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:27.962819 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:27.962874 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:27.962959 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:27.963724 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:27.964472 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:27.965739 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:27.966174 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:27.966276 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:27.968048 (+ 1772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:27.968054 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:27.968056 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:27.968086 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:27.970151 (+ 2065us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:27.970789 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:27.970793 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:27.970795 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:27.970880 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:27.971217 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:27.971220 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:27.971222 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:27.971410 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:27.971520 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:27.971833 (+ 313us) server_negotiation.cc:299] Negotiation successful
0811 19:00:27.971949 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260811 19:00:28.916720 21753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:28.916051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:42815 (local address 127.13.109.1:55443)
0811 19:00:28.916472 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:28.916590 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:42815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":89,"threads_started":1}
I20260811 19:00:28.916965 21754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:28.916489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:42815 (local address 127.13.109.3:43411)
0811 19:00:28.916764 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:28.916858 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:42815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":66,"threads_started":1}
W20260811 19:00:28.917181 21377 heartbeater.cc:644] Failed to heartbeat to 127.13.109.62:42815 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.13.109.62:42815: connect: Connection refused (error 111)
W20260811 19:00:28.917258 21679 heartbeater.cc:644] Failed to heartbeat to 127.13.109.62:42815 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.13.109.62:42815: connect: Connection refused (error 111)
I20260811 19:00:28.918927 21755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:28.918563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:42815 (local address 127.13.109.2:60383)
0811 19:00:28.918780 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:28.918853 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:42815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":58,"threads_started":1}
W20260811 19:00:28.919368 21526 heartbeater.cc:644] Failed to heartbeat to 127.13.109.62:42815 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.13.109.62:42815: connect: Connection refused (error 111)
I20260811 19:00:29.432021 21756 raft_consensus.cc:482] T 00000000000000000000000000000000 P 26408052dd7842e1a0a036cce7c682c6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 449f60a2c460491d97c80e5cd420ef6e)
I20260811 19:00:29.432269 21756 raft_consensus.cc:504] T 00000000000000000000000000000000 P 26408052dd7842e1a0a036cce7c682c6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "449f60a2c460491d97c80e5cd420ef6e" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 42815 } } peers { permanent_uuid: "26408052dd7842e1a0a036cce7c682c6" member_type: VOTER last_known_addr { host: "127.13.109.61" port: 43661 } attrs { promote: false } } peers { permanent_uuid: "adbbc0043cb14763a95172b87454443a" member_type: VOTER last_known_addr { host: "127.13.109.60" port: 34827 } attrs { promote: false } }
I20260811 19:00:29.432600 21757 raft_consensus.cc:482] T 00000000000000000000000000000000 P adbbc0043cb14763a95172b87454443a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 449f60a2c460491d97c80e5cd420ef6e)
I20260811 19:00:29.432713 21757 raft_consensus.cc:504] T 00000000000000000000000000000000 P adbbc0043cb14763a95172b87454443a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "449f60a2c460491d97c80e5cd420ef6e" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 42815 } } peers { permanent_uuid: "26408052dd7842e1a0a036cce7c682c6" member_type: VOTER last_known_addr { host: "127.13.109.61" port: 43661 } attrs { promote: false } } peers { permanent_uuid: "adbbc0043cb14763a95172b87454443a" member_type: VOTER last_known_addr { host: "127.13.109.60" port: 34827 } attrs { promote: false } }
I20260811 19:00:29.433265 21757 leader_election.cc:288] T 00000000000000000000000000000000 P adbbc0043cb14763a95172b87454443a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 449f60a2c460491d97c80e5cd420ef6e (127.13.109.62:42815), 26408052dd7842e1a0a036cce7c682c6 (127.13.109.61:43661)
I20260811 19:00:29.433454 21756 leader_election.cc:288] T 00000000000000000000000000000000 P 26408052dd7842e1a0a036cce7c682c6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 449f60a2c460491d97c80e5cd420ef6e (127.13.109.62:42815), adbbc0043cb14763a95172b87454443a (127.13.109.60:34827)
I20260811 19:00:29.434106 21759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:29.433580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:42815 (local address 127.0.0.1:50666)
0811 19:00:29.433884 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:29.433980 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:42815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":62,"threads_started":1}
W20260811 19:00:29.434310 20969 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.109.62:42815: connect: Connection refused (error 111)
I20260811 19:00:29.434432 21758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:29.433485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:42815 (local address 127.0.0.1:50650)
0811 19:00:29.434122 (+ 637us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:29.434321 (+ 199us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:42815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":568,"thread_start_us":87,"threads_started":1}
W20260811 19:00:29.434616 21172 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.109.62:42815: connect: Connection refused (error 111)
I20260811 19:00:29.435286 21759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:29.435006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:42815 (local address 127.0.0.1:50670)
0811 19:00:29.435167 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:29.435219 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:42815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
W20260811 19:00:29.435463 20969 leader_election.cc:334] T 00000000000000000000000000000000 P 26408052dd7842e1a0a036cce7c682c6 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 449f60a2c460491d97c80e5cd420ef6e (127.13.109.62:42815): Network error: Client connection negotiation failed: client connection to 127.13.109.62:42815: connect: Connection refused (error 111)
I20260811 19:00:29.435578 21758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:29.435309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:42815 (local address 127.0.0.1:50684)
0811 19:00:29.435466 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:29.435512 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:42815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
W20260811 19:00:29.435750 21172 leader_election.cc:334] T 00000000000000000000000000000000 P adbbc0043cb14763a95172b87454443a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 449f60a2c460491d97c80e5cd420ef6e (127.13.109.62:42815): Network error: Client connection negotiation failed: client connection to 127.13.109.62:42815: connect: Connection refused (error 111)
I20260811 19:00:29.438371 21761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:29.433856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52266 (local address 127.13.109.61:43661)
0811 19:00:29.434668 (+ 812us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:29.434672 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:29.434688 (+ 16us) server_negotiation.cc:407] Connection header received
0811 19:00:29.434737 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:29.434740 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:29.434786 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:29.435251 (+ 465us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:00:29.435497 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:29.436332 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:29.437359 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:29.437766 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:29.437941 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:29.438004 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:29.438167 (+ 163us) server_negotiation.cc:299] Negotiation successful
0811 19:00:29.438244 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":103,"server-negotiator.queue_time_us":742,"thread_start_us":64,"threads_started":1}
I20260811 19:00:29.438371 21763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:29.434097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.61:43661 (local address 127.0.0.1:52266)
0811 19:00:29.434350 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:29.434363 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:29.434464 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:29.434964 (+ 500us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:29.434967 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:29.434990 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:00:29.435132 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:29.435140 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:29.436447 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:29.436452 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:29.437231 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:29.437240 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:29.437936 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:29.437939 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:29.438032 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:29.438059 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 19:00:29.438151 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 19:00:29.438231 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":67,"threads_started":1}
I20260811 19:00:29.438391 21762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:29.433878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46594 (local address 127.13.109.60:34827)
0811 19:00:29.434871 (+ 993us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:29.434877 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:29.434893 (+ 16us) server_negotiation.cc:407] Connection header received
0811 19:00:29.434945 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:29.434948 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:29.435007 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:29.435089 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:00:29.435510 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:29.436249 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:29.437309 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:29.437730 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:29.437906 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:29.437972 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:29.438165 (+ 193us) server_negotiation.cc:299] Negotiation successful
0811 19:00:29.438250 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":869,"spinlock_wait_cycles":2816,"thread_start_us":62,"threads_started":1}
I20260811 19:00:29.438380 21760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:29.433642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.60:34827 (local address 127.0.0.1:46594)
0811 19:00:29.433926 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:29.433944 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:29.434095 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:29.435204 (+ 1109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:29.435207 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:29.435223 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:00:29.435375 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:29.435383 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:29.436365 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:29.436370 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:29.437173 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:29.437182 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:29.437921 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:29.437925 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:29.438030 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:29.438053 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 19:00:29.438148 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 19:00:29.438228 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201,"mutex_wait_us":3,"thread_start_us":73,"threads_started":1}
I20260811 19:00:29.438812 20993 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "adbbc0043cb14763a95172b87454443a" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "26408052dd7842e1a0a036cce7c682c6" is_pre_election: true
I20260811 19:00:29.439008 20993 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 26408052dd7842e1a0a036cce7c682c6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate adbbc0043cb14763a95172b87454443a in term 1.
I20260811 19:00:29.439378 21170 leader_election.cc:302] T 00000000000000000000000000000000 P adbbc0043cb14763a95172b87454443a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 26408052dd7842e1a0a036cce7c682c6, adbbc0043cb14763a95172b87454443a; no voters: 449f60a2c460491d97c80e5cd420ef6e
I20260811 19:00:29.439448 21193 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "26408052dd7842e1a0a036cce7c682c6" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "adbbc0043cb14763a95172b87454443a" is_pre_election: true
I20260811 19:00:29.439563 21757 raft_consensus.cc:2781] T 00000000000000000000000000000000 P adbbc0043cb14763a95172b87454443a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260811 19:00:29.439617 21757 raft_consensus.cc:482] T 00000000000000000000000000000000 P adbbc0043cb14763a95172b87454443a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 449f60a2c460491d97c80e5cd420ef6e)
I20260811 19:00:29.439670 21757 raft_consensus.cc:3037] T 00000000000000000000000000000000 P adbbc0043cb14763a95172b87454443a [term 1 FOLLOWER]: Advancing to term 2
I20260811 19:00:29.440850 21757 raft_consensus.cc:504] T 00000000000000000000000000000000 P adbbc0043cb14763a95172b87454443a [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "449f60a2c460491d97c80e5cd420ef6e" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 42815 } } peers { permanent_uuid: "26408052dd7842e1a0a036cce7c682c6" member_type: VOTER last_known_addr { host: "127.13.109.61" port: 43661 } attrs { promote: false } } peers { permanent_uuid: "adbbc0043cb14763a95172b87454443a" member_type: VOTER last_known_addr { host: "127.13.109.60" port: 34827 } attrs { promote: false } }
I20260811 19:00:29.441203 21193 raft_consensus.cc:2379] T 00000000000000000000000000000000 P adbbc0043cb14763a95172b87454443a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 26408052dd7842e1a0a036cce7c682c6 in current term 2: Already voted for candidate adbbc0043cb14763a95172b87454443a in this term.
I20260811 19:00:29.441275 21757 leader_election.cc:288] T 00000000000000000000000000000000 P adbbc0043cb14763a95172b87454443a [CANDIDATE]: Term 2 election: Requested vote from peers 449f60a2c460491d97c80e5cd420ef6e (127.13.109.62:42815), 26408052dd7842e1a0a036cce7c682c6 (127.13.109.61:43661)
I20260811 19:00:29.441649 20993 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "adbbc0043cb14763a95172b87454443a" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "26408052dd7842e1a0a036cce7c682c6"
I20260811 19:00:29.441808 20993 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 26408052dd7842e1a0a036cce7c682c6 [term 1 FOLLOWER]: Advancing to term 2
I20260811 19:00:29.441995 21763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:29.441472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:42815 (local address 127.0.0.1:50686)
0811 19:00:29.441701 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:29.441774 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:42815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129}
I20260811 19:00:29.442307 20967 leader_election.cc:302] T 00000000000000000000000000000000 P 26408052dd7842e1a0a036cce7c682c6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 26408052dd7842e1a0a036cce7c682c6; no voters: 449f60a2c460491d97c80e5cd420ef6e, adbbc0043cb14763a95172b87454443a
I20260811 19:00:29.442818 21763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:29.442510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:42815 (local address 127.0.0.1:50708)
0811 19:00:29.442688 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:29.442747 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:42815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124}
W20260811 19:00:29.442926 21172 leader_election.cc:334] T 00000000000000000000000000000000 P adbbc0043cb14763a95172b87454443a [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 449f60a2c460491d97c80e5cd420ef6e (127.13.109.62:42815): Network error: Client connection negotiation failed: client connection to 127.13.109.62:42815: connect: Connection refused (error 111)
I20260811 19:00:29.443279 20993 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 26408052dd7842e1a0a036cce7c682c6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate adbbc0043cb14763a95172b87454443a in term 2.
I20260811 19:00:29.443408 21756 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 26408052dd7842e1a0a036cce7c682c6 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 19:00:29.443630 21170 leader_election.cc:302] T 00000000000000000000000000000000 P adbbc0043cb14763a95172b87454443a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 26408052dd7842e1a0a036cce7c682c6, adbbc0043cb14763a95172b87454443a; no voters: 449f60a2c460491d97c80e5cd420ef6e
I20260811 19:00:29.443802 21757 raft_consensus.cc:2781] T 00000000000000000000000000000000 P adbbc0043cb14763a95172b87454443a [term 2 FOLLOWER]: Leader election won for term 2
I20260811 19:00:29.444052 21757 raft_consensus.cc:686] T 00000000000000000000000000000000 P adbbc0043cb14763a95172b87454443a [term 2 LEADER]: Becoming Leader. State: Replica: adbbc0043cb14763a95172b87454443a, State: Running, Role: LEADER
I20260811 19:00:29.444340 21757 consensus_queue.cc:237] T 00000000000000000000000000000000 P adbbc0043cb14763a95172b87454443a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "449f60a2c460491d97c80e5cd420ef6e" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 42815 } } peers { permanent_uuid: "26408052dd7842e1a0a036cce7c682c6" member_type: VOTER last_known_addr { host: "127.13.109.61" port: 43661 } attrs { promote: false } } peers { permanent_uuid: "adbbc0043cb14763a95172b87454443a" member_type: VOTER last_known_addr { host: "127.13.109.60" port: 34827 } attrs { promote: false } }
I20260811 19:00:29.445650 21766 sys_catalog.cc:455] T 00000000000000000000000000000000 P adbbc0043cb14763a95172b87454443a [sys.catalog]: SysCatalogTable state changed. Reason: New leader adbbc0043cb14763a95172b87454443a. Latest consensus state: current_term: 2 leader_uuid: "adbbc0043cb14763a95172b87454443a" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "449f60a2c460491d97c80e5cd420ef6e" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 42815 } } peers { permanent_uuid: "26408052dd7842e1a0a036cce7c682c6" member_type: VOTER last_known_addr { host: "127.13.109.61" port: 43661 } attrs { promote: false } } peers { permanent_uuid: "adbbc0043cb14763a95172b87454443a" member_type: VOTER last_known_addr { host: "127.13.109.60" port: 34827 } attrs { promote: false } } }
I20260811 19:00:29.445748 21766 sys_catalog.cc:458] T 00000000000000000000000000000000 P adbbc0043cb14763a95172b87454443a [sys.catalog]: This master's current role is: LEADER
I20260811 19:00:29.446115 21768 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:00:29.448469 21768 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:00:29.448889 21768 catalog_manager.cc:1259] Loaded cluster ID: 2362e23819bd411a8fb9f457d6f232c5
I20260811 19:00:29.448961 21768 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:00:29.449538 21768 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:00:29.449999 21768 catalog_manager.cc:5879] T 00000000000000000000000000000000 P adbbc0043cb14763a95172b87454443a: Loaded TSK: 0
I20260811 19:00:29.450711 21768 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 19:00:29.507298 21182 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46580:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 19:00:29.510617 21182 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 19:00:29.513587 21763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:29.513330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:42815 (local address 127.0.0.1:50722)
0811 19:00:29.513455 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:29.513518 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:42815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260811 19:00:29.513739 20993 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 26408052dd7842e1a0a036cce7c682c6 [term 2 FOLLOWER]: Refusing update from remote peer adbbc0043cb14763a95172b87454443a: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260811 19:00:29.514345 21763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:29.514094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:42815 (local address 127.0.0.1:50738)
0811 19:00:29.514202 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:29.514250 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:42815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260811 19:00:29.514358 21757 consensus_queue.cc:1035] T 00000000000000000000000000000000 P adbbc0043cb14763a95172b87454443a [LEADER]: Connected to new peer: Peer: permanent_uuid: "26408052dd7842e1a0a036cce7c682c6" member_type: VOTER last_known_addr { host: "127.13.109.61" port: 43661 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260811 19:00:29.514551 21172 consensus_peers.cc:487] T 00000000000000000000000000000000 P adbbc0043cb14763a95172b87454443a -> Peer 449f60a2c460491d97c80e5cd420ef6e (127.13.109.62:42815): Couldn't send request to peer 449f60a2c460491d97c80e5cd420ef6e. Status: Network error: Client connection negotiation failed: client connection to 127.13.109.62:42815: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 19:00:29.515962 21756 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26408052dd7842e1a0a036cce7c682c6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader adbbc0043cb14763a95172b87454443a. Latest consensus state: current_term: 2 leader_uuid: "adbbc0043cb14763a95172b87454443a" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "449f60a2c460491d97c80e5cd420ef6e" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 42815 } } peers { permanent_uuid: "26408052dd7842e1a0a036cce7c682c6" member_type: VOTER last_known_addr { host: "127.13.109.61" port: 43661 } attrs { promote: false } } peers { permanent_uuid: "adbbc0043cb14763a95172b87454443a" member_type: VOTER last_known_addr { host: "127.13.109.60" port: 34827 } attrs { promote: false } } }
I20260811 19:00:29.516124 21756 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26408052dd7842e1a0a036cce7c682c6 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 19:00:29.517335 21766 sys_catalog.cc:455] T 00000000000000000000000000000000 P adbbc0043cb14763a95172b87454443a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "adbbc0043cb14763a95172b87454443a" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "449f60a2c460491d97c80e5cd420ef6e" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 42815 } } peers { permanent_uuid: "26408052dd7842e1a0a036cce7c682c6" member_type: VOTER last_known_addr { host: "127.13.109.61" port: 43661 } attrs { promote: false } } peers { permanent_uuid: "adbbc0043cb14763a95172b87454443a" member_type: VOTER last_known_addr { host: "127.13.109.60" port: 34827 } attrs { promote: false } } }
I20260811 19:00:29.517470 21766 sys_catalog.cc:458] T 00000000000000000000000000000000 P adbbc0043cb14763a95172b87454443a [sys.catalog]: This master's current role is: LEADER
I20260811 19:00:29.517887 21756 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26408052dd7842e1a0a036cce7c682c6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "adbbc0043cb14763a95172b87454443a" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "449f60a2c460491d97c80e5cd420ef6e" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 42815 } } peers { permanent_uuid: "26408052dd7842e1a0a036cce7c682c6" member_type: VOTER last_known_addr { host: "127.13.109.61" port: 43661 } attrs { promote: false } } peers { permanent_uuid: "adbbc0043cb14763a95172b87454443a" member_type: VOTER last_known_addr { host: "127.13.109.60" port: 34827 } attrs { promote: false } } }
I20260811 19:00:29.518018 21756 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26408052dd7842e1a0a036cce7c682c6 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 19:00:29.528162 21763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:29.524153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.2:32839 (local address 127.0.0.1:46600)
0811 19:00:29.524317 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:29.524336 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:29.524423 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:29.524832 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:29.524837 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:29.524873 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:00:29.524999 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:29.525007 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:29.526129 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:29.526135 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:29.526885 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:29.526893 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:29.527736 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:29.527739 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:29.527839 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:29.527869 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 19:00:29.527911 (+ 42us) client_negotiation.cc:240] Negotiation successful
0811 19:00:29.527980 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93}
I20260811 19:00:29.528689 21776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:29.524250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46600 (local address 127.13.109.2:32839)
0811 19:00:29.524546 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:29.524552 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:29.524588 (+ 36us) server_negotiation.cc:407] Connection header received
0811 19:00:29.524648 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:29.524651 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:29.524705 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:29.525116 (+ 411us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:00:29.525248 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:29.525998 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:29.527012 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:29.527600 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:29.528379 (+ 779us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:29.528445 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:29.528482 (+ 37us) server_negotiation.cc:299] Negotiation successful
0811 19:00:29.528554 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":96,"threads_started":1}
I20260811 19:00:29.528918 21777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:29.524792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35524 (local address 127.13.109.3:43545)
0811 19:00:29.525134 (+ 342us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:29.525139 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:29.525155 (+ 16us) server_negotiation.cc:407] Connection header received
0811 19:00:29.525208 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:29.525211 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:29.525257 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:29.525615 (+ 358us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:00:29.525709 (+ 94us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:29.526271 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:29.527579 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:29.528200 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:29.528310 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:29.528375 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:29.528721 (+ 346us) server_negotiation.cc:299] Negotiation successful
0811 19:00:29.528799 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":249,"thread_start_us":99,"threads_started":1}
I20260811 19:00:29.528910 21758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:29.524679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.3:43545 (local address 127.0.0.1:35524)
0811 19:00:29.524832 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:29.524844 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:29.524908 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:29.525388 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:29.525392 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:29.525409 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:00:29.525515 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:29.525520 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:29.526395 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:29.526400 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:29.527450 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:29.527458 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:29.528494 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:29.528497 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:29.528588 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:29.528608 (+ 20us) client_negotiation.cc:769] Sending connection context
0811 19:00:29.528702 (+ 94us) client_negotiation.cc:240] Negotiation successful
0811 19:00:29.528782 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88}
I20260811 19:00:29.529467 21455 tablet_service.cc:1467] Processing CreateTablet for tablet 92dc925b54024dfc95d74fbd7c1fc0ab (DEFAULT_TABLE table=test-table [id=7174a7a213eb4203a8d3dcfb1b710dc3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:00:29.529846 21455 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92dc925b54024dfc95d74fbd7c1fc0ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:00:29.530339 21778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:29.525123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.1:38457 (local address 127.0.0.1:39292)
0811 19:00:29.525451 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:29.525464 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:29.525574 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:29.526869 (+ 1295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:29.526872 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:29.526891 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:00:29.526984 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:29.526989 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:29.528081 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:29.528084 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:29.529245 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:29.529254 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:29.529944 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:29.529948 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:29.530039 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:29.530061 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 19:00:29.530154 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 19:00:29.530218 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":89,"threads_started":1}
I20260811 19:00:29.530376 21779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:29.525249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39292 (local address 127.13.109.1:38457)
0811 19:00:29.526551 (+ 1302us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:29.526556 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:29.526569 (+ 13us) server_negotiation.cc:407] Connection header received
0811 19:00:29.526619 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:29.526622 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:29.526670 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:29.526756 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:00:29.527249 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:29.527966 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:29.529366 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:29.529846 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:29.529943 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:29.530001 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:29.530172 (+ 171us) server_negotiation.cc:299] Negotiation successful
0811 19:00:29.530255 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1211,"thread_start_us":73,"threads_started":1}
I20260811 19:00:29.530882 21305 tablet_service.cc:1467] Processing CreateTablet for tablet 92dc925b54024dfc95d74fbd7c1fc0ab (DEFAULT_TABLE table=test-table [id=7174a7a213eb4203a8d3dcfb1b710dc3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:00:29.531183 21305 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92dc925b54024dfc95d74fbd7c1fc0ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:00:29.532482 21780 tablet_bootstrap.cc:492] T 92dc925b54024dfc95d74fbd7c1fc0ab P adf5bc4ddc704fec824189012815251a: Bootstrap starting.
I20260811 19:00:29.533303 21603 tablet_service.cc:1467] Processing CreateTablet for tablet 92dc925b54024dfc95d74fbd7c1fc0ab (DEFAULT_TABLE table=test-table [id=7174a7a213eb4203a8d3dcfb1b710dc3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:00:29.533622 21603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92dc925b54024dfc95d74fbd7c1fc0ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:00:29.533623 21781 tablet_bootstrap.cc:492] T 92dc925b54024dfc95d74fbd7c1fc0ab P 2c9427fe9258493f9465751fef6bb43a: Bootstrap starting.
I20260811 19:00:29.533666 21780 tablet_bootstrap.cc:654] T 92dc925b54024dfc95d74fbd7c1fc0ab P adf5bc4ddc704fec824189012815251a: Neither blocks nor log segments found. Creating new log.
I20260811 19:00:29.534735 21781 tablet_bootstrap.cc:654] T 92dc925b54024dfc95d74fbd7c1fc0ab P 2c9427fe9258493f9465751fef6bb43a: Neither blocks nor log segments found. Creating new log.
I20260811 19:00:29.534994 21780 tablet_bootstrap.cc:492] T 92dc925b54024dfc95d74fbd7c1fc0ab P adf5bc4ddc704fec824189012815251a: No bootstrap required, opened a new log
I20260811 19:00:29.535102 21780 ts_tablet_manager.cc:1397] T 92dc925b54024dfc95d74fbd7c1fc0ab P adf5bc4ddc704fec824189012815251a: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 19:00:29.535521 21780 raft_consensus.cc:348] T 92dc925b54024dfc95d74fbd7c1fc0ab P adf5bc4ddc704fec824189012815251a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adf5bc4ddc704fec824189012815251a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 32839 } } peers { permanent_uuid: "2c9427fe9258493f9465751fef6bb43a" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 38457 } } peers { permanent_uuid: "20f4f9eaa6844d8b9466235cf97ec8ce" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43545 } }
I20260811 19:00:29.535645 21780 raft_consensus.cc:374] T 92dc925b54024dfc95d74fbd7c1fc0ab P adf5bc4ddc704fec824189012815251a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:00:29.535682 21780 raft_consensus.cc:729] T 92dc925b54024dfc95d74fbd7c1fc0ab P adf5bc4ddc704fec824189012815251a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: adf5bc4ddc704fec824189012815251a, State: Initialized, Role: FOLLOWER
I20260811 19:00:29.535820 21780 consensus_queue.cc:260] T 92dc925b54024dfc95d74fbd7c1fc0ab P adf5bc4ddc704fec824189012815251a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adf5bc4ddc704fec824189012815251a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 32839 } } peers { permanent_uuid: "2c9427fe9258493f9465751fef6bb43a" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 38457 } } peers { permanent_uuid: "20f4f9eaa6844d8b9466235cf97ec8ce" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43545 } }
I20260811 19:00:29.536233 21780 ts_tablet_manager.cc:1428] T 92dc925b54024dfc95d74fbd7c1fc0ab P adf5bc4ddc704fec824189012815251a: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260811 19:00:29.536252 21785 tablet_bootstrap.cc:492] T 92dc925b54024dfc95d74fbd7c1fc0ab P 20f4f9eaa6844d8b9466235cf97ec8ce: Bootstrap starting.
I20260811 19:00:29.537082 21786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:29.536696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:42815 (local address 127.13.109.2:45567)
0811 19:00:29.536939 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:29.537004 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:42815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":102,"threads_started":1}
I20260811 19:00:29.537263 21781 tablet_bootstrap.cc:492] T 92dc925b54024dfc95d74fbd7c1fc0ab P 2c9427fe9258493f9465751fef6bb43a: No bootstrap required, opened a new log
I20260811 19:00:29.537359 21781 ts_tablet_manager.cc:1397] T 92dc925b54024dfc95d74fbd7c1fc0ab P 2c9427fe9258493f9465751fef6bb43a: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260811 19:00:29.537612 21785 tablet_bootstrap.cc:654] T 92dc925b54024dfc95d74fbd7c1fc0ab P 20f4f9eaa6844d8b9466235cf97ec8ce: Neither blocks nor log segments found. Creating new log.
I20260811 19:00:29.537935 21781 raft_consensus.cc:348] T 92dc925b54024dfc95d74fbd7c1fc0ab P 2c9427fe9258493f9465751fef6bb43a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adf5bc4ddc704fec824189012815251a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 32839 } } peers { permanent_uuid: "2c9427fe9258493f9465751fef6bb43a" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 38457 } } peers { permanent_uuid: "20f4f9eaa6844d8b9466235cf97ec8ce" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43545 } }
I20260811 19:00:29.538064 21781 raft_consensus.cc:374] T 92dc925b54024dfc95d74fbd7c1fc0ab P 2c9427fe9258493f9465751fef6bb43a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:00:29.538100 21781 raft_consensus.cc:729] T 92dc925b54024dfc95d74fbd7c1fc0ab P 2c9427fe9258493f9465751fef6bb43a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c9427fe9258493f9465751fef6bb43a, State: Initialized, Role: FOLLOWER
I20260811 19:00:29.538242 21781 consensus_queue.cc:260] T 92dc925b54024dfc95d74fbd7c1fc0ab P 2c9427fe9258493f9465751fef6bb43a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adf5bc4ddc704fec824189012815251a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 32839 } } peers { permanent_uuid: "2c9427fe9258493f9465751fef6bb43a" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 38457 } } peers { permanent_uuid: "20f4f9eaa6844d8b9466235cf97ec8ce" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43545 } }
I20260811 19:00:29.538612 21781 ts_tablet_manager.cc:1428] T 92dc925b54024dfc95d74fbd7c1fc0ab P 2c9427fe9258493f9465751fef6bb43a: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260811 19:00:29.539650 21790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:29.539264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:42815 (local address 127.13.109.1:40523)
0811 19:00:29.539507 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:29.539567 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:42815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":107,"threads_started":1}
I20260811 19:00:29.540759 21525 heartbeater.cc:503] Master 127.13.109.60:34827 was elected leader, sending a full tablet report...
I20260811 19:00:29.542060 21786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:29.541712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:42815 (local address 127.13.109.2:36293)
0811 19:00:29.541923 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:29.541986 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:42815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123}
I20260811 19:00:29.542682 21785 tablet_bootstrap.cc:492] T 92dc925b54024dfc95d74fbd7c1fc0ab P 20f4f9eaa6844d8b9466235cf97ec8ce: No bootstrap required, opened a new log
I20260811 19:00:29.542768 21785 ts_tablet_manager.cc:1397] T 92dc925b54024dfc95d74fbd7c1fc0ab P 20f4f9eaa6844d8b9466235cf97ec8ce: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.002s
I20260811 19:00:29.543176 21785 raft_consensus.cc:348] T 92dc925b54024dfc95d74fbd7c1fc0ab P 20f4f9eaa6844d8b9466235cf97ec8ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adf5bc4ddc704fec824189012815251a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 32839 } } peers { permanent_uuid: "2c9427fe9258493f9465751fef6bb43a" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 38457 } } peers { permanent_uuid: "20f4f9eaa6844d8b9466235cf97ec8ce" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43545 } }
I20260811 19:00:29.543270 21785 raft_consensus.cc:374] T 92dc925b54024dfc95d74fbd7c1fc0ab P 20f4f9eaa6844d8b9466235cf97ec8ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:00:29.543291 21785 raft_consensus.cc:729] T 92dc925b54024dfc95d74fbd7c1fc0ab P 20f4f9eaa6844d8b9466235cf97ec8ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20f4f9eaa6844d8b9466235cf97ec8ce, State: Initialized, Role: FOLLOWER
I20260811 19:00:29.543422 21785 consensus_queue.cc:260] T 92dc925b54024dfc95d74fbd7c1fc0ab P 20f4f9eaa6844d8b9466235cf97ec8ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adf5bc4ddc704fec824189012815251a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 32839 } } peers { permanent_uuid: "2c9427fe9258493f9465751fef6bb43a" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 38457 } } peers { permanent_uuid: "20f4f9eaa6844d8b9466235cf97ec8ce" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43545 } }
I20260811 19:00:29.543715 21785 ts_tablet_manager.cc:1428] T 92dc925b54024dfc95d74fbd7c1fc0ab P 20f4f9eaa6844d8b9466235cf97ec8ce: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260811 19:00:29.544653 21792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:29.544113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:42815 (local address 127.13.109.3:32947)
0811 19:00:29.544478 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:29.544546 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:42815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":100,"threads_started":1}
I20260811 19:00:29.545450 21677 heartbeater.cc:503] Master 127.13.109.60:34827 was elected leader, sending a full tablet report...
I20260811 19:00:29.545773 21793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:29.545309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:42815 (local address 127.13.109.3:44919)
0811 19:00:29.545612 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:29.545671 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:42815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":102,"threads_started":1}
I20260811 19:00:29.589205 21784 raft_consensus.cc:482] T 92dc925b54024dfc95d74fbd7c1fc0ab P adf5bc4ddc704fec824189012815251a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:00:29.589349 21784 raft_consensus.cc:504] T 92dc925b54024dfc95d74fbd7c1fc0ab P adf5bc4ddc704fec824189012815251a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adf5bc4ddc704fec824189012815251a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 32839 } } peers { permanent_uuid: "2c9427fe9258493f9465751fef6bb43a" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 38457 } } peers { permanent_uuid: "20f4f9eaa6844d8b9466235cf97ec8ce" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43545 } }
I20260811 19:00:29.589920 21784 leader_election.cc:288] T 92dc925b54024dfc95d74fbd7c1fc0ab P adf5bc4ddc704fec824189012815251a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c9427fe9258493f9465751fef6bb43a (127.13.109.1:38457), 20f4f9eaa6844d8b9466235cf97ec8ce (127.13.109.3:43545)
I20260811 19:00:29.590408 21327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92dc925b54024dfc95d74fbd7c1fc0ab" candidate_uuid: "adf5bc4ddc704fec824189012815251a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9427fe9258493f9465751fef6bb43a" is_pre_election: true
I20260811 19:00:29.590484 21627 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92dc925b54024dfc95d74fbd7c1fc0ab" candidate_uuid: "adf5bc4ddc704fec824189012815251a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20f4f9eaa6844d8b9466235cf97ec8ce" is_pre_election: true
I20260811 19:00:29.590597 21327 raft_consensus.cc:2454] T 92dc925b54024dfc95d74fbd7c1fc0ab P 2c9427fe9258493f9465751fef6bb43a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate adf5bc4ddc704fec824189012815251a in term 0.
I20260811 19:00:29.590678 21627 raft_consensus.cc:2454] T 92dc925b54024dfc95d74fbd7c1fc0ab P 20f4f9eaa6844d8b9466235cf97ec8ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate adf5bc4ddc704fec824189012815251a in term 0.
I20260811 19:00:29.590971 21405 leader_election.cc:302] T 92dc925b54024dfc95d74fbd7c1fc0ab P adf5bc4ddc704fec824189012815251a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9427fe9258493f9465751fef6bb43a, adf5bc4ddc704fec824189012815251a; no voters:
I20260811 19:00:29.591176 21784 raft_consensus.cc:2781] T 92dc925b54024dfc95d74fbd7c1fc0ab P adf5bc4ddc704fec824189012815251a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 19:00:29.591238 21784 raft_consensus.cc:482] T 92dc925b54024dfc95d74fbd7c1fc0ab P adf5bc4ddc704fec824189012815251a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 19:00:29.591280 21784 raft_consensus.cc:3037] T 92dc925b54024dfc95d74fbd7c1fc0ab P adf5bc4ddc704fec824189012815251a [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:00:29.592381 21784 raft_consensus.cc:504] T 92dc925b54024dfc95d74fbd7c1fc0ab P adf5bc4ddc704fec824189012815251a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adf5bc4ddc704fec824189012815251a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 32839 } } peers { permanent_uuid: "2c9427fe9258493f9465751fef6bb43a" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 38457 } } peers { permanent_uuid: "20f4f9eaa6844d8b9466235cf97ec8ce" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43545 } }
I20260811 19:00:29.592837 21784 leader_election.cc:288] T 92dc925b54024dfc95d74fbd7c1fc0ab P adf5bc4ddc704fec824189012815251a [CANDIDATE]: Term 1 election: Requested vote from peers 2c9427fe9258493f9465751fef6bb43a (127.13.109.1:38457), 20f4f9eaa6844d8b9466235cf97ec8ce (127.13.109.3:43545)
I20260811 19:00:29.593246 21327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92dc925b54024dfc95d74fbd7c1fc0ab" candidate_uuid: "adf5bc4ddc704fec824189012815251a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9427fe9258493f9465751fef6bb43a"
I20260811 19:00:29.593314 21627 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92dc925b54024dfc95d74fbd7c1fc0ab" candidate_uuid: "adf5bc4ddc704fec824189012815251a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20f4f9eaa6844d8b9466235cf97ec8ce"
I20260811 19:00:29.593380 21327 raft_consensus.cc:3037] T 92dc925b54024dfc95d74fbd7c1fc0ab P 2c9427fe9258493f9465751fef6bb43a [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:00:29.593480 21627 raft_consensus.cc:3037] T 92dc925b54024dfc95d74fbd7c1fc0ab P 20f4f9eaa6844d8b9466235cf97ec8ce [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:00:29.594594 21327 raft_consensus.cc:2454] T 92dc925b54024dfc95d74fbd7c1fc0ab P 2c9427fe9258493f9465751fef6bb43a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate adf5bc4ddc704fec824189012815251a in term 1.
I20260811 19:00:29.594956 21627 raft_consensus.cc:2454] T 92dc925b54024dfc95d74fbd7c1fc0ab P 20f4f9eaa6844d8b9466235cf97ec8ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate adf5bc4ddc704fec824189012815251a in term 1.
I20260811 19:00:29.595047 21405 leader_election.cc:302] T 92dc925b54024dfc95d74fbd7c1fc0ab P adf5bc4ddc704fec824189012815251a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9427fe9258493f9465751fef6bb43a, adf5bc4ddc704fec824189012815251a; no voters:
I20260811 19:00:29.595288 21784 raft_consensus.cc:2781] T 92dc925b54024dfc95d74fbd7c1fc0ab P adf5bc4ddc704fec824189012815251a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:00:29.595526 21784 raft_consensus.cc:686] T 92dc925b54024dfc95d74fbd7c1fc0ab P adf5bc4ddc704fec824189012815251a [term 1 LEADER]: Becoming Leader. State: Replica: adf5bc4ddc704fec824189012815251a, State: Running, Role: LEADER
I20260811 19:00:29.595707 21784 consensus_queue.cc:237] T 92dc925b54024dfc95d74fbd7c1fc0ab P adf5bc4ddc704fec824189012815251a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adf5bc4ddc704fec824189012815251a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 32839 } } peers { permanent_uuid: "2c9427fe9258493f9465751fef6bb43a" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 38457 } } peers { permanent_uuid: "20f4f9eaa6844d8b9466235cf97ec8ce" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43545 } }
I20260811 19:00:29.596305 21786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:29.596026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:42815 (local address 127.13.109.2:56121)
0811 19:00:29.596188 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:29.596237 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:42815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260811 19:00:29.597589 21184 catalog_manager.cc:5495] T 92dc925b54024dfc95d74fbd7c1fc0ab P adf5bc4ddc704fec824189012815251a reported cstate change: term changed from 0 to 1, leader changed from <none> to adf5bc4ddc704fec824189012815251a (127.13.109.2). New cstate: current_term: 1 leader_uuid: "adf5bc4ddc704fec824189012815251a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adf5bc4ddc704fec824189012815251a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 32839 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c9427fe9258493f9465751fef6bb43a" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 38457 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20f4f9eaa6844d8b9466235cf97ec8ce" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43545 } health_report { overall_health: UNKNOWN } } }
I20260811 19:00:29.611755 21776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:29.608771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46614 (local address 127.13.109.2:32839)
0811 19:00:29.608944 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:29.608949 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:29.608987 (+ 38us) server_negotiation.cc:407] Connection header received
0811 19:00:29.609033 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:29.609035 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:29.609081 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:29.609139 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:00:29.609362 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:29.609975 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:29.610772 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:29.610986 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:29.611075 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:29.611281 (+ 206us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:00:29.611353 (+ 72us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:00:29.611407 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:29.611503 (+ 96us) server_negotiation.cc:299] Negotiation successful
0811 19:00:29.611559 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 19:00:29.614141 21627 raft_consensus.cc:1261] T 92dc925b54024dfc95d74fbd7c1fc0ab P 20f4f9eaa6844d8b9466235cf97ec8ce [term 1 FOLLOWER]: Refusing update from remote peer adf5bc4ddc704fec824189012815251a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:00:29.614156 21327 raft_consensus.cc:1261] T 92dc925b54024dfc95d74fbd7c1fc0ab P 2c9427fe9258493f9465751fef6bb43a [term 1 FOLLOWER]: Refusing update from remote peer adf5bc4ddc704fec824189012815251a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:00:29.614656 21784 consensus_queue.cc:1035] T 92dc925b54024dfc95d74fbd7c1fc0ab P adf5bc4ddc704fec824189012815251a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9427fe9258493f9465751fef6bb43a" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 38457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:00:29.614830 21794 consensus_queue.cc:1035] T 92dc925b54024dfc95d74fbd7c1fc0ab P adf5bc4ddc704fec824189012815251a [LEADER]: Connected to new peer: Peer: permanent_uuid: "20f4f9eaa6844d8b9466235cf97ec8ce" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:00:29.616961 21376 heartbeater.cc:503] Master 127.13.109.60:34827 was elected leader, sending a full tablet report...
I20260811 19:00:29.617645 21790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:29.617295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:42815 (local address 127.13.109.1:50193)
0811 19:00:29.617467 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:29.617532 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:42815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260811 19:00:29.618363 21793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:29.618003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:42815 (local address 127.13.109.3:43763)
0811 19:00:29.618233 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:29.618291 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:42815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165}
I20260811 19:00:29.620265 21786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:29.619955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:42815 (local address 127.13.109.2:39409)
0811 19:00:29.620117 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:29.620188 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:42815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260811 19:00:29.620901 21786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:29.620681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:42815 (local address 127.13.109.2:53995)
0811 19:00:29.620791 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:29.620842 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:42815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260811 19:00:29.624424 21793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:29.622443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:42815 (local address 127.13.109.3:55405)
0811 19:00:29.624265 (+ 1822us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:29.624326 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:42815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1745}
I20260811 19:00:29.624643 21790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:29.624362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:42815 (local address 127.13.109.1:51695)
0811 19:00:29.624513 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:29.624570 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:42815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260811 19:00:29.625698 21185 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46580:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:00:29.625905 21185 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46580:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:00:29.630260 21185 catalog_manager.cc:5782] T 00000000000000000000000000000000 P adbbc0043cb14763a95172b87454443a: Sending DeleteTablet for 3 replicas of tablet 92dc925b54024dfc95d74fbd7c1fc0ab
I20260811 19:00:29.631155 21455 tablet_service.cc:1514] Processing DeleteTablet for tablet 92dc925b54024dfc95d74fbd7c1fc0ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:00:29 UTC) from {username='oryx', principal='oryx/127.13.109.60@KRBTEST.COM'} at 127.0.0.1:46600
I20260811 19:00:29.631246 21603 tablet_service.cc:1514] Processing DeleteTablet for tablet 92dc925b54024dfc95d74fbd7c1fc0ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:00:29 UTC) from {username='oryx', principal='oryx/127.13.109.60@KRBTEST.COM'} at 127.0.0.1:35524
I20260811 19:00:29.631248 21305 tablet_service.cc:1514] Processing DeleteTablet for tablet 92dc925b54024dfc95d74fbd7c1fc0ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:00:29 UTC) from {username='oryx', principal='oryx/127.13.109.60@KRBTEST.COM'} at 127.0.0.1:39292
I20260811 19:00:29.631596 21807 tablet_replica.cc:330] T 92dc925b54024dfc95d74fbd7c1fc0ab P 20f4f9eaa6844d8b9466235cf97ec8ce: stopping tablet replica
I20260811 19:00:29.631664 21808 tablet_replica.cc:330] T 92dc925b54024dfc95d74fbd7c1fc0ab P 2c9427fe9258493f9465751fef6bb43a: stopping tablet replica
I20260811 19:00:29.631747 21807 raft_consensus.cc:2229] T 92dc925b54024dfc95d74fbd7c1fc0ab P 20f4f9eaa6844d8b9466235cf97ec8ce [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:00:29.631803 21808 raft_consensus.cc:2229] T 92dc925b54024dfc95d74fbd7c1fc0ab P 2c9427fe9258493f9465751fef6bb43a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:00:29.631902 21807 raft_consensus.cc:2258] T 92dc925b54024dfc95d74fbd7c1fc0ab P 20f4f9eaa6844d8b9466235cf97ec8ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:00:29.631950 21808 raft_consensus.cc:2258] T 92dc925b54024dfc95d74fbd7c1fc0ab P 2c9427fe9258493f9465751fef6bb43a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:00:29.632215 21806 tablet_replica.cc:330] T 92dc925b54024dfc95d74fbd7c1fc0ab P adf5bc4ddc704fec824189012815251a: stopping tablet replica
I20260811 19:00:29.632406 21806 raft_consensus.cc:2229] T 92dc925b54024dfc95d74fbd7c1fc0ab P adf5bc4ddc704fec824189012815251a [term 1 LEADER]: Raft consensus shutting down.
I20260811 19:00:29.632524 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 21143
I20260811 19:00:29.635748 21806 raft_consensus.cc:2258] T 92dc925b54024dfc95d74fbd7c1fc0ab P adf5bc4ddc704fec824189012815251a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:00:29.636641 21806 ts_tablet_manager.cc:1905] T 92dc925b54024dfc95d74fbd7c1fc0ab P adf5bc4ddc704fec824189012815251a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:00:29.638537 21807 ts_tablet_manager.cc:1905] T 92dc925b54024dfc95d74fbd7c1fc0ab P 20f4f9eaa6844d8b9466235cf97ec8ce: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:00:29.640416 21806 ts_tablet_manager.cc:1918] T 92dc925b54024dfc95d74fbd7c1fc0ab P adf5bc4ddc704fec824189012815251a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:00:29.640522 21806 log.cc:1196] T 92dc925b54024dfc95d74fbd7c1fc0ab P adf5bc4ddc704fec824189012815251a: Deleting WAL directory at /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/ts-1/wal/wals/92dc925b54024dfc95d74fbd7c1fc0ab
I20260811 19:00:29.640836 21807 ts_tablet_manager.cc:1918] T 92dc925b54024dfc95d74fbd7c1fc0ab P 20f4f9eaa6844d8b9466235cf97ec8ce: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:00:29.640863 21806 ts_tablet_manager.cc:1939] T 92dc925b54024dfc95d74fbd7c1fc0ab P adf5bc4ddc704fec824189012815251a: Deleting consensus metadata
I20260811 19:00:29.640941 21807 log.cc:1196] T 92dc925b54024dfc95d74fbd7c1fc0ab P 20f4f9eaa6844d8b9466235cf97ec8ce: Deleting WAL directory at /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474792496492-13748-0/minicluster-data/ts-2/wal/wals/92dc925b54024dfc95d74fbd7c1fc0ab
I20260811 19:00:29.641198 21807 ts_tablet_manager.cc:1939] T 92dc925b54024dfc95d74fbd7c1fc0ab P 20f4f9eaa6844d8b9466235cf97ec8ce: Deleting consensus metadata
I20260811 19:00:29.641877 21173 catalog_manager.cc:4841] TS adf5bc4ddc704fec824189012815251a (127.13.109.2:32839): tablet 92dc925b54024dfc95d74fbd7c1fc0ab (table test-table [id=7174a7a213eb4203a8d3dcfb1b710dc3]) successfully deleted
I20260811 19:00:29.642182 21173 catalog_manager.cc:4841] TS 20f4f9eaa6844d8b9466235cf97ec8ce (127.13.109.3:43545): tablet 92dc925b54024dfc95d74fbd7c1fc0ab (table test-table [id=7174a7a213eb4203a8d3dcfb1b710dc3]) successfully deleted
I20260811 19:00:29.644901 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 21387
I20260811 19:00:29.645293 21778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:29.644929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.1:38457 (local address 127.0.0.1:39300)
0811 19:00:29.645105 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:29.645181 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.1:38457: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260811 19:00:29.646731 21171 catalog_manager.cc:4584] TS 2c9427fe9258493f9465751fef6bb43a (127.13.109.1:38457): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 92dc925b54024dfc95d74fbd7c1fc0ab: Network error: Client connection negotiation failed: client connection to 127.13.109.1:38457: connect: Connection refused (error 111)
I20260811 19:00:29.655572 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 21532
I20260811 19:00:29.664924 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 20839
I20260811 19:00:29.674263 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 20949
2026-08-11T19:00:29Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (5626 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:00:29 dist-test-slave-0kls krb5kdc[21811](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:00:29 dist-test-slave-0kls krb5kdc[21811](info): set up 2 sockets
Aug 11 19:00:29 dist-test-slave-0kls krb5kdc[21811](info): commencing operation
krb5kdc: starting...
W20260811 19:00:30.776858 13748 mini_kdc.cc:121] Time spent starting KDC: real 1.074s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/krb5kdc/test-user.keytab.
Aug 11 19:00:30 dist-test-slave-0kls krb5kdc[21811](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474830, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:00:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:00:30Z Disabled control of system clock
WARNING: no policy specified for oryx/127.13.109.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.109.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.109.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/krb5kdc/oryx_127.13.109.62.keytab.
Entry for principal oryx/127.13.109.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/krb5kdc/oryx_127.13.109.62.keytab.
I20260811 19:00:30.916923 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.62:37395
--webserver_interface=127.13.109.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/krb5kdc/oryx_127.13.109.62.keytab
--principal=oryx/127.13.109.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:40507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.109.62:37395
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:00:31.037556 21827 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:00:31.037983 21827 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:00:31.038095 21827 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:00:31.042418 21827 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:00:31.042507 21827 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 19:00:31.042539 21827 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:00:31.042578 21827 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:00:31.042614 21827 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:00:31.042654 21827 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:00:31.048348 21827 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:40507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.109.62:37395
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/krb5kdc/oryx_127.13.109.62.keytab
--principal=oryx/127.13.109.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.62:37395
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.109.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21827
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:00:31.049589 21827 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:00:31.050808 21827 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:00:31.060173 21836 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:31.060352 21827 server_base.cc:1034] running on GCE node
W20260811 19:00:31.060207 21834 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:31.060173 21833 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:31.061182 21827 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:00:31.062695 21827 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:00:31.063917 21827 hybrid_clock.cc:648] HybridClock initialized: now 1786474831063896 us; error 33 us; skew 500 ppm
Aug 11 19:00:31 dist-test-slave-0kls krb5kdc[21811](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474831, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.109.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:00:31.066936 21827 init.cc:377] Logged in from keytab as oryx/127.13.109.62@KRBTEST.COM (short username oryx)
I20260811 19:00:31.067957 21827 webserver.cc:460] Webserver started at http://127.13.109.62:34879/ using document root <none> and password file <none>
I20260811 19:00:31.068557 21827 fs_manager.cc:359] Metadata directory not provided
I20260811 19:00:31.068658 21827 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:00:31.068912 21827 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:00:31.071099 21827 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/minicluster-data/master-0/data/instance:
uuid: "517db3110b44426982fea15cbe648df8"
format_stamp: "Formatted at 2026-08-11 19:00:31 on dist-test-slave-0kls"
server_key: "1ca6e90725e6e0e5994742e987eb7812"
server_key_iv: "036b54eaeb93f1a17062d271a61461cc"
server_key_version: "encryptionkey@0"
I20260811 19:00:31.071709 21827 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/minicluster-data/master-0/wal/instance:
uuid: "517db3110b44426982fea15cbe648df8"
format_stamp: "Formatted at 2026-08-11 19:00:31 on dist-test-slave-0kls"
server_key: "1ca6e90725e6e0e5994742e987eb7812"
server_key_iv: "036b54eaeb93f1a17062d271a61461cc"
server_key_version: "encryptionkey@0"
I20260811 19:00:31.075784 21827 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 19:00:31.078328 21843 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:31.079584 21827 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 19:00:31.079747 21827 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/minicluster-data/master-0/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/minicluster-data/master-0/wal
uuid: "517db3110b44426982fea15cbe648df8"
format_stamp: "Formatted at 2026-08-11 19:00:31 on dist-test-slave-0kls"
server_key: "1ca6e90725e6e0e5994742e987eb7812"
server_key_iv: "036b54eaeb93f1a17062d271a61461cc"
server_key_version: "encryptionkey@0"
I20260811 19:00:31.079870 21827 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:00:31.094561 21827 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:00:31.098099 21827 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:00:31.098340 21827 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:00:31.107584 21827 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.62:37395
I20260811 19:00:31.107610 21908 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.62:37395 every 8 connection(s)
I20260811 19:00:31.108709 21827 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
I20260811 19:00:31.111977 21909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:00:31.115530 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 21827
I20260811 19:00:31.115705 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/minicluster-data/master-0/wal/instance
I20260811 19:00:31.116036 13748 external_mini_cluster.cc:1428] Setting key 368cc32d0fcccacfb36d68c3adc15238
I20260811 19:00:31.119066 21909 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 517db3110b44426982fea15cbe648df8: Bootstrap starting.
I20260811 19:00:31.122107 21909 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 517db3110b44426982fea15cbe648df8: Neither blocks nor log segments found. Creating new log.
I20260811 19:00:31.123123 21909 log.cc:824] T 00000000000000000000000000000000 P 517db3110b44426982fea15cbe648df8: Log is configured to *not* fsync() on all Append() calls
I20260811 19:00:31.126042 21909 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 517db3110b44426982fea15cbe648df8: No bootstrap required, opened a new log
Aug 11 19:00:31 dist-test-slave-0kls krb5kdc[21811](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474830, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM
I20260811 19:00:31.131070 21909 raft_consensus.cc:348] T 00000000000000000000000000000000 P 517db3110b44426982fea15cbe648df8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "517db3110b44426982fea15cbe648df8" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 37395 } }
I20260811 19:00:31.131443 21909 raft_consensus.cc:374] T 00000000000000000000000000000000 P 517db3110b44426982fea15cbe648df8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:00:31.131510 21909 raft_consensus.cc:729] T 00000000000000000000000000000000 P 517db3110b44426982fea15cbe648df8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 517db3110b44426982fea15cbe648df8, State: Initialized, Role: FOLLOWER
I20260811 19:00:31.132046 21909 consensus_queue.cc:260] T 00000000000000000000000000000000 P 517db3110b44426982fea15cbe648df8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "517db3110b44426982fea15cbe648df8" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 37395 } }
I20260811 19:00:31.132230 21909 raft_consensus.cc:388] T 00000000000000000000000000000000 P 517db3110b44426982fea15cbe648df8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:00:31.132299 21909 raft_consensus.cc:482] T 00000000000000000000000000000000 P 517db3110b44426982fea15cbe648df8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:00:31.132400 21909 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 517db3110b44426982fea15cbe648df8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:00:31.132747 21913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:31.117696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60700 (local address 127.13.109.62:37395)
0811 19:00:31.118264 (+ 568us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:31.118282 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:31.118324 (+ 42us) server_negotiation.cc:407] Connection header received
0811 19:00:31.119006 (+ 682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:31.119024 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:31.119482 (+ 458us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:31.119830 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:31.120895 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:31.122008 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:31.123250 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:31.123921 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:31.124113 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:31.127088 (+ 2975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:31.127095 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:31.127115 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:31.127152 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:31.129242 (+ 2090us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:31.129727 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:31.129735 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:31.129743 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:31.129933 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:31.130234 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:31.130237 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:31.130239 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:31.130826 (+ 587us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:31.131042 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:31.131297 (+ 255us) server_negotiation.cc:299] Negotiation successful
0811 19:00:31.131641 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":154,"threads_started":1}
I20260811 19:00:31.133697 21909 raft_consensus.cc:504] T 00000000000000000000000000000000 P 517db3110b44426982fea15cbe648df8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "517db3110b44426982fea15cbe648df8" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 37395 } }
I20260811 19:00:31.134112 21909 leader_election.cc:302] T 00000000000000000000000000000000 P 517db3110b44426982fea15cbe648df8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 517db3110b44426982fea15cbe648df8; no voters:
I20260811 19:00:31.134483 21909 leader_election.cc:288] T 00000000000000000000000000000000 P 517db3110b44426982fea15cbe648df8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:00:31.134692 21916 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 517db3110b44426982fea15cbe648df8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:00:31.134972 21916 raft_consensus.cc:686] T 00000000000000000000000000000000 P 517db3110b44426982fea15cbe648df8 [term 1 LEADER]: Becoming Leader. State: Replica: 517db3110b44426982fea15cbe648df8, State: Running, Role: LEADER
I20260811 19:00:31.135344 21916 consensus_queue.cc:237] T 00000000000000000000000000000000 P 517db3110b44426982fea15cbe648df8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "517db3110b44426982fea15cbe648df8" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 37395 } }
I20260811 19:00:31.135854 21909 sys_catalog.cc:564] T 00000000000000000000000000000000 P 517db3110b44426982fea15cbe648df8 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:00:31.137290 21919 sys_catalog.cc:455] T 00000000000000000000000000000000 P 517db3110b44426982fea15cbe648df8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 517db3110b44426982fea15cbe648df8. Latest consensus state: current_term: 1 leader_uuid: "517db3110b44426982fea15cbe648df8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "517db3110b44426982fea15cbe648df8" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 37395 } } }
I20260811 19:00:31.137425 21919 sys_catalog.cc:458] T 00000000000000000000000000000000 P 517db3110b44426982fea15cbe648df8 [sys.catalog]: This master's current role is: LEADER
I20260811 19:00:31.137688 21918 sys_catalog.cc:455] T 00000000000000000000000000000000 P 517db3110b44426982fea15cbe648df8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "517db3110b44426982fea15cbe648df8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "517db3110b44426982fea15cbe648df8" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 37395 } } }
I20260811 19:00:31.137817 21918 sys_catalog.cc:458] T 00000000000000000000000000000000 P 517db3110b44426982fea15cbe648df8 [sys.catalog]: This master's current role is: LEADER
I20260811 19:00:31.138196 21928 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:00:31.141539 21928 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:00:31.147176 21928 catalog_manager.cc:1347] Generated new cluster ID: 436e78e7f6a04a569a2c4f96eb57bf28
I20260811 19:00:31.147261 21928 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:00:31.160804 21928 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:00:31.161975 21928 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:00:31.171913 21928 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 517db3110b44426982fea15cbe648df8: Generated new TSK 0
I20260811 19:00:31.172731 21928 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.13.109.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.109.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.109.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/krb5kdc/oryx_127.13.109.1.keytab.
Entry for principal oryx/127.13.109.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/krb5kdc/oryx_127.13.109.1.keytab.
I20260811 19:00:31.214003 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.1:0
--local_ip_for_outbound_sockets=127.13.109.1
--webserver_interface=127.13.109.1
--webserver_port=0
--tserver_master_addrs=127.13.109.62:37395
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/krb5kdc/oryx_127.13.109.1.keytab
--principal=oryx/127.13.109.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:40507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:00:31.333716 21940 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:00:31.334034 21940 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:00:31.334080 21940 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:00:31.338466 21940 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:00:31.338536 21940 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:00:31.338585 21940 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 19:00:31.338620 21940 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.1
I20260811 19:00:31.344316 21940 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:40507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/krb5kdc/oryx_127.13.109.1.keytab
--principal=oryx/127.13.109.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.109.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.109.62:37395
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21940
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.1
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:00:31.345427 21940 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:00:31.346485 21940 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:00:31.357188 21945 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:31.357290 21940 server_base.cc:1034] running on GCE node
W20260811 19:00:31.357188 21948 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:31.357208 21946 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:31.357977 21940 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:00:31.358716 21940 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:00:31.359887 21940 hybrid_clock.cc:648] HybridClock initialized: now 1786474831359868 us; error 31 us; skew 500 ppm
Aug 11 19:00:31 dist-test-slave-0kls krb5kdc[21811](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474831, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.109.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:00:31.362938 21940 init.cc:377] Logged in from keytab as oryx/127.13.109.1@KRBTEST.COM (short username oryx)
I20260811 19:00:31.363950 21940 webserver.cc:460] Webserver started at http://127.13.109.1:37317/ using document root <none> and password file <none>
I20260811 19:00:31.364558 21940 fs_manager.cc:359] Metadata directory not provided
I20260811 19:00:31.364661 21940 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:00:31.364903 21940 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:00:31.367103 21940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/minicluster-data/ts-0/data/instance:
uuid: "3bf50bf4e77b45298676246bb4db1765"
format_stamp: "Formatted at 2026-08-11 19:00:31 on dist-test-slave-0kls"
server_key: "e46bbd3da8708f478922db6b250f1dd5"
server_key_iv: "8de9669eabfa203ea475f959f905aa5f"
server_key_version: "encryptionkey@0"
I20260811 19:00:31.367693 21940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/minicluster-data/ts-0/wal/instance:
uuid: "3bf50bf4e77b45298676246bb4db1765"
format_stamp: "Formatted at 2026-08-11 19:00:31 on dist-test-slave-0kls"
server_key: "e46bbd3da8708f478922db6b250f1dd5"
server_key_iv: "8de9669eabfa203ea475f959f905aa5f"
server_key_version: "encryptionkey@0"
I20260811 19:00:31.371795 21940 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 19:00:31.374487 21955 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:31.375752 21940 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 19:00:31.375893 21940 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/minicluster-data/ts-0/wal
uuid: "3bf50bf4e77b45298676246bb4db1765"
format_stamp: "Formatted at 2026-08-11 19:00:31 on dist-test-slave-0kls"
server_key: "e46bbd3da8708f478922db6b250f1dd5"
server_key_iv: "8de9669eabfa203ea475f959f905aa5f"
server_key_version: "encryptionkey@0"
I20260811 19:00:31.376015 21940 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:00:31.390723 21940 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:00:31.394685 21940 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:00:31.394925 21940 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:00:31.396606 21940 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:00:31.396668 21940 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:31.396747 21940 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:00:31.396781 21940 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:31.407987 21940 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.1:36147
I20260811 19:00:31.408066 22078 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.1:36147 every 8 connection(s)
Aug 11 19:00:31 dist-test-slave-0kls krb5kdc[21811](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474831, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.109.1@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM
I20260811 19:00:31.409127 21940 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
I20260811 19:00:31.412177 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 21940
I20260811 19:00:31.412361 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/minicluster-data/ts-0/wal/instance
I20260811 19:00:31.412742 13748 external_mini_cluster.cc:1428] Setting key ce419717825aa56da308f1410f2537ff
I20260811 19:00:31.412859 21913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:31.397643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.1:48381 (local address 127.13.109.62:37395)
0811 19:00:31.397822 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:31.397830 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:31.399112 (+ 1282us) server_negotiation.cc:407] Connection header received
0811 19:00:31.400392 (+ 1280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:31.400399 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:31.400478 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:31.400597 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:31.402434 (+ 1837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:31.403390 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:31.405163 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:31.405423 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:31.405557 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:31.408903 (+ 3346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:31.408907 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:31.408910 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:31.408950 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:31.410560 (+ 1610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:31.411097 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:31.411101 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:31.411103 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:31.411198 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:31.411583 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:31.411587 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:31.411588 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:31.411802 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:31.411917 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:31.412508 (+ 591us) server_negotiation.cc:299] Negotiation successful
0811 19:00:31.412670 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 19:00:31.413754 21965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:31.398306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:37395 (local address 127.13.109.1:48381)
0811 19:00:31.398907 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:31.398956 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:31.400090 (+ 1134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:31.400906 (+ 816us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:31.400916 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:31.401491 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:31.402254 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:31.402272 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:31.403530 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:31.403534 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:31.405017 (+ 1483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:31.405028 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:31.405661 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:31.405668 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:31.405975 (+ 307us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:31.406781 (+ 806us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:31.406802 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:31.408697 (+ 1895us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:00:31.410684 (+ 1987us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:31.410689 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:31.410702 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:31.410968 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:31.411299 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:31.411302 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:31.411304 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:31.411463 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:31.411916 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:00:31.411924 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:00:31.412192 (+ 268us) client_negotiation.cc:769] Sending connection context
0811 19:00:31.412420 (+ 228us) client_negotiation.cc:240] Negotiation successful
0811 19:00:31.412724 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":173,"threads_started":1}
I20260811 19:00:31.415177 22080 heartbeater.cc:347] Connected to a master server at 127.13.109.62:37395
I20260811 19:00:31.415326 22080 heartbeater.cc:464] Registering TS with master...
I20260811 19:00:31.415923 22080 heartbeater.cc:511] Master 127.13.109.62:37395 requested a full tablet report, sending...
I20260811 19:00:31.417598 21859 ts_manager.cc:194] Registered new tserver with Master: 3bf50bf4e77b45298676246bb4db1765 (127.13.109.1:36147)
I20260811 19:00:31.419042 21859 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.109.1@KRBTEST.COM'} at 127.13.109.1:48381
WARNING: no policy specified for oryx/127.13.109.2@KRBTEST.COM; defaulting to no policy
I20260811 19:00:31.425818 21913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:31.417713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.1:33023 (local address 127.13.109.62:37395)
0811 19:00:31.417870 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:31.417873 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:31.418322 (+ 449us) server_negotiation.cc:407] Connection header received
0811 19:00:31.418446 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:31.418449 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:31.418495 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:31.418580 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:31.419228 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:31.419958 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:31.420964 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:31.421191 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:31.421301 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:31.422519 (+ 1218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:31.422522 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:31.422525 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:31.422553 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:31.423995 (+ 1442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:31.424467 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:31.424471 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:31.424473 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:31.424557 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:31.424893 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:31.424897 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:31.424898 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:31.425100 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:31.425214 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:31.425486 (+ 272us) server_negotiation.cc:299] Negotiation successful
0811 19:00:31.425605 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 19:00:31.425890 22088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:31.417866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:37395 (local address 127.13.109.1:33023)
0811 19:00:31.418212 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:31.418229 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:31.418344 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:31.418635 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:31.418640 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:31.418894 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:31.419060 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:31.419071 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:31.420114 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:31.420117 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:31.420842 (+ 725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:31.420852 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:31.421311 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:31.421314 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:31.421407 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:31.422028 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:31.422039 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:31.422379 (+ 340us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:00:31.424133 (+ 1754us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:31.424138 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:31.424140 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:31.424355 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:31.424651 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:31.424655 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:31.424657 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:31.424766 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:31.425204 (+ 438us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:00:31.425207 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:00:31.425314 (+ 107us) client_negotiation.cc:769] Sending connection context
0811 19:00:31.425427 (+ 113us) client_negotiation.cc:240] Negotiation successful
0811 19:00:31.425602 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"mutex_wait_us":22,"thread_start_us":117,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.109.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.109.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/krb5kdc/oryx_127.13.109.2.keytab.
Entry for principal oryx/127.13.109.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/krb5kdc/oryx_127.13.109.2.keytab.
I20260811 19:00:31.446676 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.2:0
--local_ip_for_outbound_sockets=127.13.109.2
--webserver_interface=127.13.109.2
--webserver_port=0
--tserver_master_addrs=127.13.109.62:37395
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/krb5kdc/oryx_127.13.109.2.keytab
--principal=oryx/127.13.109.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:40507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:00:31.566620 22090 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:00:31.566922 22090 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:00:31.566972 22090 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:00:31.571194 22090 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:00:31.571271 22090 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:00:31.571322 22090 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 19:00:31.571419 22090 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.2
I20260811 19:00:31.576865 22090 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:40507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/krb5kdc/oryx_127.13.109.2.keytab
--principal=oryx/127.13.109.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.109.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.109.62:37395
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22090
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.2
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:00:31.578135 22090 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:00:31.579265 22090 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:00:31.589955 22096 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:31.589931 22095 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:31.590070 22090 server_base.cc:1034] running on GCE node
W20260811 19:00:31.589920 22099 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:31.590725 22090 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:00:31.591470 22090 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:00:31.592691 22090 hybrid_clock.cc:648] HybridClock initialized: now 1786474831592670 us; error 33 us; skew 500 ppm
Aug 11 19:00:31 dist-test-slave-0kls krb5kdc[21811](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474831, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.109.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:00:31.595814 22090 init.cc:377] Logged in from keytab as oryx/127.13.109.2@KRBTEST.COM (short username oryx)
I20260811 19:00:31.596798 22090 webserver.cc:460] Webserver started at http://127.13.109.2:44297/ using document root <none> and password file <none>
I20260811 19:00:31.597414 22090 fs_manager.cc:359] Metadata directory not provided
I20260811 19:00:31.597514 22090 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:00:31.597774 22090 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:00:31.600127 22090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/minicluster-data/ts-1/data/instance:
uuid: "01dababe80fc41b48b6db09ba9cf2daa"
format_stamp: "Formatted at 2026-08-11 19:00:31 on dist-test-slave-0kls"
server_key: "0801729b92fc34fb74a688bc12f49c00"
server_key_iv: "23b15aa7344fb691c97adc39772e2ba1"
server_key_version: "encryptionkey@0"
I20260811 19:00:31.600770 22090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/minicluster-data/ts-1/wal/instance:
uuid: "01dababe80fc41b48b6db09ba9cf2daa"
format_stamp: "Formatted at 2026-08-11 19:00:31 on dist-test-slave-0kls"
server_key: "0801729b92fc34fb74a688bc12f49c00"
server_key_iv: "23b15aa7344fb691c97adc39772e2ba1"
server_key_version: "encryptionkey@0"
I20260811 19:00:31.605079 22090 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260811 19:00:31.607825 22106 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:31.609146 22090 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 19:00:31.609293 22090 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/minicluster-data/ts-1/wal
uuid: "01dababe80fc41b48b6db09ba9cf2daa"
format_stamp: "Formatted at 2026-08-11 19:00:31 on dist-test-slave-0kls"
server_key: "0801729b92fc34fb74a688bc12f49c00"
server_key_iv: "23b15aa7344fb691c97adc39772e2ba1"
server_key_version: "encryptionkey@0"
I20260811 19:00:31.609421 22090 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:00:31.632493 22090 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:00:31.636152 22090 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:00:31.636399 22090 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:00:31.638239 22090 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:00:31.638307 22090 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:31.638360 22090 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:00:31.638389 22090 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 19:00:31 dist-test-slave-0kls krb5kdc[21811](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474831, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.109.2@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM
I20260811 19:00:31.649969 22090 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.2:34689
I20260811 19:00:31.650049 22230 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.2:34689 every 8 connection(s)
I20260811 19:00:31.651094 22090 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
I20260811 19:00:31.652424 21913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:31.638935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.2:57551 (local address 127.13.109.62:37395)
0811 19:00:31.639099 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:31.639106 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:31.639960 (+ 854us) server_negotiation.cc:407] Connection header received
0811 19:00:31.640917 (+ 957us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:31.640920 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:31.640986 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:31.641092 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:31.642440 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:31.643350 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:31.644892 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:31.645272 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:31.645402 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:31.648604 (+ 3202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:31.648611 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:31.648618 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:31.648653 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:31.650271 (+ 1618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:31.650771 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:31.650775 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:31.650777 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:31.650866 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:31.651232 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:31.651235 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:31.651237 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:31.651479 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:31.651602 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:31.652188 (+ 586us) server_negotiation.cc:299] Negotiation successful
0811 19:00:31.652299 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 19:00:31.653257 22116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:31.639327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:37395 (local address 127.13.109.2:57551)
0811 19:00:31.639790 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:31.639842 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:31.640708 (+ 866us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:31.641370 (+ 662us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:31.641376 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:31.641747 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:31.642287 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:31.642308 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:31.643509 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:31.643512 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:31.644743 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:31.644755 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:31.645464 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:31.645468 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:31.645700 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:31.646434 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:31.646458 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:31.648413 (+ 1955us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:00:31.650393 (+ 1980us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:31.650397 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:31.650408 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:31.650663 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:31.650960 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:31.650962 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:31.650963 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:31.651089 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:31.651608 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:00:31.651624 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:00:31.651903 (+ 279us) client_negotiation.cc:769] Sending connection context
0811 19:00:31.652115 (+ 212us) client_negotiation.cc:240] Negotiation successful
0811 19:00:31.652421 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":130,"threads_started":1}
I20260811 19:00:31.654436 22231 heartbeater.cc:347] Connected to a master server at 127.13.109.62:37395
I20260811 19:00:31.654573 22231 heartbeater.cc:464] Registering TS with master...
I20260811 19:00:31.655047 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 22090
I20260811 19:00:31.655174 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/minicluster-data/ts-1/wal/instance
I20260811 19:00:31.655174 22231 heartbeater.cc:511] Master 127.13.109.62:37395 requested a full tablet report, sending...
I20260811 19:00:31.655541 13748 external_mini_cluster.cc:1428] Setting key 222b58b1b8d61ed15e8ca29638deb62a
I20260811 19:00:31.656317 21860 ts_manager.cc:194] Registered new tserver with Master: 01dababe80fc41b48b6db09ba9cf2daa (127.13.109.2:34689)
I20260811 19:00:31.657140 21860 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.109.2@KRBTEST.COM'} at 127.13.109.2:57551
I20260811 19:00:31.665169 22239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:31.656767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:37395 (local address 127.13.109.2:52815)
0811 19:00:31.657571 (+ 804us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:31.657587 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:31.657723 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:31.658065 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:31.658068 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:31.658278 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:31.658382 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:31.658387 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:31.659264 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:31.659268 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:31.659835 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:31.659844 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:31.660349 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:31.660354 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:31.660479 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:31.661002 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:31.661022 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:31.661498 (+ 476us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:00:31.663603 (+ 2105us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:31.663606 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:31.663608 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:31.663807 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:31.664110 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:31.664112 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:31.664114 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:31.664189 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:31.664748 (+ 559us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:00:31.664750 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:00:31.664829 (+ 79us) client_negotiation.cc:769] Sending connection context
0811 19:00:31.664907 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 19:00:31.665021 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":707,"thread_start_us":95,"threads_started":1}
I20260811 19:00:31.665397 21913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:31.656832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.2:52815 (local address 127.13.109.62:37395)
0811 19:00:31.657436 (+ 604us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:31.657443 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:31.657656 (+ 213us) server_negotiation.cc:407] Connection header received
0811 19:00:31.657893 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:31.657896 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:31.657942 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:31.658047 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:31.658506 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:31.659135 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:31.659959 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:31.660208 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:31.660314 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:31.661683 (+ 1369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:31.661689 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:31.661695 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:31.661729 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:31.663481 (+ 1752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:31.663895 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:31.663902 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:31.663906 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:31.664006 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:31.664295 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:31.664300 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:31.664302 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:31.664640 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:31.664756 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:31.665008 (+ 252us) server_negotiation.cc:299] Negotiation successful
0811 19:00:31.665150 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":514}
WARNING: no policy specified for oryx/127.13.109.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.109.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.109.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/krb5kdc/oryx_127.13.109.3.keytab.
Entry for principal oryx/127.13.109.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/krb5kdc/oryx_127.13.109.3.keytab.
I20260811 19:00:31.689663 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.3:0
--local_ip_for_outbound_sockets=127.13.109.3
--webserver_interface=127.13.109.3
--webserver_port=0
--tserver_master_addrs=127.13.109.62:37395
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/krb5kdc/oryx_127.13.109.3.keytab
--principal=oryx/127.13.109.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:40507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:00:31.810916 22242 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:00:31.811250 22242 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:00:31.811333 22242 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:00:31.815951 22242 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:00:31.816051 22242 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:00:31.816125 22242 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 19:00:31.816207 22242 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.3
I20260811 19:00:31.822054 22242 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:40507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/krb5kdc/oryx_127.13.109.3.keytab
--principal=oryx/127.13.109.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.109.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.109.62:37395
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22242
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.3
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:00:31.823349 22242 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:00:31.824491 22242 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 19:00:31.834712 22242 server_base.cc:1034] running on GCE node
W20260811 19:00:31.834697 22249 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:31.834726 22248 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:31.834699 22251 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:31.835506 22242 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:00:31.836210 22242 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:00:31.837369 22242 hybrid_clock.cc:648] HybridClock initialized: now 1786474831837341 us; error 60 us; skew 500 ppm
Aug 11 19:00:31 dist-test-slave-0kls krb5kdc[21811](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474831, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.109.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:00:31.840571 22242 init.cc:377] Logged in from keytab as oryx/127.13.109.3@KRBTEST.COM (short username oryx)
I20260811 19:00:31.841775 22242 webserver.cc:460] Webserver started at http://127.13.109.3:43523/ using document root <none> and password file <none>
I20260811 19:00:31.842548 22242 fs_manager.cc:359] Metadata directory not provided
I20260811 19:00:31.842643 22242 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:00:31.842872 22242 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:00:31.845033 22242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/minicluster-data/ts-2/data/instance:
uuid: "aa3ef1ab1dce4ff0876fed61e6719733"
format_stamp: "Formatted at 2026-08-11 19:00:31 on dist-test-slave-0kls"
server_key: "fbeec36afe3adaf9e054056c6acd423e"
server_key_iv: "0dfaf7e417ae9ec531b372b9bc91bb3b"
server_key_version: "encryptionkey@0"
I20260811 19:00:31.845649 22242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/minicluster-data/ts-2/wal/instance:
uuid: "aa3ef1ab1dce4ff0876fed61e6719733"
format_stamp: "Formatted at 2026-08-11 19:00:31 on dist-test-slave-0kls"
server_key: "fbeec36afe3adaf9e054056c6acd423e"
server_key_iv: "0dfaf7e417ae9ec531b372b9bc91bb3b"
server_key_version: "encryptionkey@0"
I20260811 19:00:31.849776 22242 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260811 19:00:31.852531 22259 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:31.853886 22242 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 19:00:31.854018 22242 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/minicluster-data/ts-2/wal
uuid: "aa3ef1ab1dce4ff0876fed61e6719733"
format_stamp: "Formatted at 2026-08-11 19:00:31 on dist-test-slave-0kls"
server_key: "fbeec36afe3adaf9e054056c6acd423e"
server_key_iv: "0dfaf7e417ae9ec531b372b9bc91bb3b"
server_key_version: "encryptionkey@0"
I20260811 19:00:31.854148 22242 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:00:31.875697 22242 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:00:31.879041 22242 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:00:31.879263 22242 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:00:31.880920 22242 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:00:31.880993 22242 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:31.881068 22242 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:00:31.881115 22242 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 19:00:31 dist-test-slave-0kls krb5kdc[21811](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474831, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.109.3@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM
I20260811 19:00:31.892737 22242 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.3:44129
I20260811 19:00:31.892783 22385 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.3:44129 every 8 connection(s)
I20260811 19:00:31.893910 22242 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
I20260811 19:00:31.896072 21913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:31.881867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.3:44319 (local address 127.13.109.62:37395)
0811 19:00:31.882045 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:31.882052 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:31.882909 (+ 857us) server_negotiation.cc:407] Connection header received
0811 19:00:31.883993 (+ 1084us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:31.883998 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:31.884077 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:31.884169 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:31.886119 (+ 1950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:31.886742 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:31.888090 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:31.888321 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:31.888459 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:31.891976 (+ 3517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:31.891980 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:31.891983 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:31.892017 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:31.893701 (+ 1684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:31.894407 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:31.894411 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:31.894413 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:31.894522 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:31.894971 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:31.894975 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:31.894976 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:31.895247 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:31.895336 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:31.895830 (+ 494us) server_negotiation.cc:299] Negotiation successful
0811 19:00:31.895945 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 19:00:31.896960 22268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:31.882265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:37395 (local address 127.13.109.3:44319)
0811 19:00:31.882738 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:31.882789 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:31.883724 (+ 935us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:31.884630 (+ 906us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:31.884639 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:31.885159 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:31.885950 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:31.885974 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:31.886890 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:31.886893 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:31.887939 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:31.887950 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:31.888523 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:31.888527 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:31.888724 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:31.889436 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:31.889458 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:31.891743 (+ 2285us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:00:31.893891 (+ 2148us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:31.893898 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:31.893914 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:31.894267 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:31.894658 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:31.894661 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:31.894663 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:31.894836 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:31.895348 (+ 512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:00:31.895353 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:00:31.895610 (+ 257us) client_negotiation.cc:769] Sending connection context
0811 19:00:31.895764 (+ 154us) client_negotiation.cc:240] Negotiation successful
0811 19:00:31.896041 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":141,"threads_started":1}
I20260811 19:00:31.897665 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 22242
I20260811 19:00:31.897852 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/minicluster-data/ts-2/wal/instance
I20260811 19:00:31.898264 13748 external_mini_cluster.cc:1428] Setting key d1c4e940d410f0d3ca7e2f4640e76814
I20260811 19:00:31.898659 22387 heartbeater.cc:347] Connected to a master server at 127.13.109.62:37395
I20260811 19:00:31.898836 22387 heartbeater.cc:464] Registering TS with master...
I20260811 19:00:31.899567 22387 heartbeater.cc:511] Master 127.13.109.62:37395 requested a full tablet report, sending...
I20260811 19:00:31.900709 21859 ts_manager.cc:194] Registered new tserver with Master: aa3ef1ab1dce4ff0876fed61e6719733 (127.13.109.3:44129)
I20260811 19:00:31.901490 21859 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.109.3@KRBTEST.COM'} at 127.13.109.3:44319
I20260811 19:00:31.908629 21913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:31.901229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.3:57457 (local address 127.13.109.62:37395)
0811 19:00:31.901351 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:31.901355 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:31.901550 (+ 195us) server_negotiation.cc:407] Connection header received
0811 19:00:31.901674 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:31.901677 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:31.901725 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:31.901851 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:31.902453 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:31.903038 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:31.904029 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:31.904254 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:31.904331 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:31.905416 (+ 1085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:31.905419 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:31.905421 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:31.905444 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:31.906943 (+ 1499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:31.907432 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:31.907435 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:31.907436 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:31.907524 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:31.907821 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:31.907823 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:31.907824 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:31.908031 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:31.908112 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:31.908384 (+ 272us) server_negotiation.cc:299] Negotiation successful
0811 19:00:31.908485 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260811 19:00:31.908660 22393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:31.901118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:37395 (local address 127.13.109.3:57457)
0811 19:00:31.901467 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:31.901479 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:31.901566 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:31.901889 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:31.901892 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:31.902158 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:31.902303 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:31.902311 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:31.903174 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:31.903178 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:31.903924 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:31.903934 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:31.904354 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:31.904357 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:31.904425 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:31.904922 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:31.904934 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:31.905282 (+ 348us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:00:31.907063 (+ 1781us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:31.907066 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:31.907067 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:31.907307 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:31.907624 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:31.907627 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:31.907628 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:31.907733 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:31.908122 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:00:31.908124 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:00:31.908231 (+ 107us) client_negotiation.cc:769] Sending connection context
0811 19:00:31.908326 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 19:00:31.908455 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"mutex_wait_us":1,"thread_start_us":104,"threads_started":1}
I20260811 19:00:31.912892 13748 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 19:00:31 dist-test-slave-0kls krb5kdc[21811](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474831, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 19:00:31 dist-test-slave-0kls krb5kdc[21811](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:00:31 dist-test-slave-0kls krb5kdc[21811](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
W20260811 19:00:31.940101 21913 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:00:31.933422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60716 (local address 127.13.109.62:37395)
0811 19:00:31.933578 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:31.933582 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:31.933595 (+ 13us) server_negotiation.cc:407] Connection header received
0811 19:00:31.933639 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:31.933641 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:31.933682 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:31.933761 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:31.934588 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:31.935119 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:31.936075 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:31.936289 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:31.936392 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:31.939931 (+ 3539us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60716: BlockingRecv error: recv got EOF from 127.0.0.1:60716 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 11 19:00:31 dist-test-slave-0kls krb5kdc[21811](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474831, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM
I20260811 19:00:31.950104 21913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:31.941707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60720 (local address 127.13.109.62:37395)
0811 19:00:31.941893 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:31.941898 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:31.942022 (+ 124us) server_negotiation.cc:407] Connection header received
0811 19:00:31.942217 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:31.942220 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:31.942263 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:31.942348 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:31.942973 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:31.943481 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:31.944291 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:31.944510 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:31.944642 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:31.947000 (+ 2358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:31.947003 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:31.947006 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:31.947031 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:31.948372 (+ 1341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:31.948823 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:31.948827 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:31.948829 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:31.948906 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:31.949194 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:31.949198 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:31.949200 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:31.949393 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:31.949537 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:31.949780 (+ 243us) server_negotiation.cc:299] Negotiation successful
0811 19:00:31.949927 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260811 19:00:31.953522 21859 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60720:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 19:00:31.957607 21859 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 19:00:31.974350 22410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:31.967695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.1:36147 (local address 127.0.0.1:39064)
0811 19:00:31.968109 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:31.968179 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:31.968327 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:31.969223 (+ 896us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:31.969229 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:31.969258 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:00:31.969427 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:31.969435 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:31.971161 (+ 1726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:31.971167 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:31.972629 (+ 1462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:31.972639 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:31.973630 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:31.973634 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:31.973755 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:31.973831 (+ 76us) client_negotiation.cc:769] Sending connection context
0811 19:00:31.973957 (+ 126us) client_negotiation.cc:240] Negotiation successful
0811 19:00:31.974128 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"mutex_wait_us":132,"thread_start_us":98,"threads_started":1}
I20260811 19:00:31.974352 22413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:31.968457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.3:44129 (local address 127.0.0.1:59116)
0811 19:00:31.969242 (+ 785us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:31.969286 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:31.969382 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:31.970133 (+ 751us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:31.970136 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:31.970155 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:00:31.970245 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:31.970251 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:31.971244 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:31.971247 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:31.972685 (+ 1438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:31.972697 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:31.973873 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:31.973876 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:31.973986 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:31.974009 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 19:00:31.974063 (+ 54us) client_negotiation.cc:240] Negotiation successful
0811 19:00:31.974176 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":721,"mutex_wait_us":78,"thread_start_us":112,"threads_started":1}
I20260811 19:00:31.974525 22411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:31.967865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39064 (local address 127.13.109.1:36147)
0811 19:00:31.968680 (+ 815us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:31.968686 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:31.968710 (+ 24us) server_negotiation.cc:407] Connection header received
0811 19:00:31.968804 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:31.968811 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:31.969041 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:31.969604 (+ 563us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:00:31.969833 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:31.971020 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:31.972785 (+ 1765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:31.973468 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:31.973631 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:31.973848 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:31.974264 (+ 416us) server_negotiation.cc:299] Negotiation successful
0811 19:00:31.974373 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":676,"thread_start_us":87,"threads_started":1}
I20260811 19:00:31.974414 22414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:31.968616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55168 (local address 127.13.109.2:34689)
0811 19:00:31.968962 (+ 346us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:31.968974 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:31.968996 (+ 22us) server_negotiation.cc:407] Connection header received
0811 19:00:31.969069 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:31.969074 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:31.969485 (+ 411us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:31.969930 (+ 445us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:00:31.970078 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:31.971177 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:31.972844 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:31.973700 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:31.973930 (+ 230us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:31.974110 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:31.974169 (+ 59us) server_negotiation.cc:299] Negotiation successful
0811 19:00:31.974269 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":250,"thread_start_us":127,"threads_started":1}
I20260811 19:00:31.974942 22412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:31.968145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.2:34689 (local address 127.0.0.1:55168)
0811 19:00:31.968450 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:31.968466 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:31.968548 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:31.969638 (+ 1090us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:31.969641 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:31.969660 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:00:31.969755 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:31.969761 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:31.971301 (+ 1540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:31.971304 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:31.972685 (+ 1381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:31.972697 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:31.973873 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:31.973876 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:31.974013 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:31.974035 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 19:00:31.974095 (+ 60us) client_negotiation.cc:240] Negotiation successful
0811 19:00:31.974152 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"mutex_wait_us":75,"thread_start_us":128,"threads_started":1}
I20260811 19:00:31.975203 22415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:31.968915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59116 (local address 127.13.109.3:44129)
0811 19:00:31.969631 (+ 716us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:31.969643 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:31.969667 (+ 24us) server_negotiation.cc:407] Connection header received
0811 19:00:31.969749 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:31.969754 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:31.969977 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:31.970155 (+ 178us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:00:31.970343 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:31.971116 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:31.972844 (+ 1728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:31.973686 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:31.974694 (+ 1008us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:31.974867 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:31.974919 (+ 52us) server_negotiation.cc:299] Negotiation successful
0811 19:00:31.975036 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":616,"thread_start_us":108,"threads_started":1}
I20260811 19:00:31.977752 22161 tablet_service.cc:1467] Processing CreateTablet for tablet 5dbc6f482ecd4c46a761422b1715dedc (DEFAULT_TABLE table=test-table [id=c57a42539620488ba7c4ca2a7087ca31]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:00:31.977743 22010 tablet_service.cc:1467] Processing CreateTablet for tablet 5dbc6f482ecd4c46a761422b1715dedc (DEFAULT_TABLE table=test-table [id=c57a42539620488ba7c4ca2a7087ca31]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:00:31.978852 22010 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5dbc6f482ecd4c46a761422b1715dedc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:00:31.978869 22161 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5dbc6f482ecd4c46a761422b1715dedc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:00:31.982250 22316 tablet_service.cc:1467] Processing CreateTablet for tablet 5dbc6f482ecd4c46a761422b1715dedc (DEFAULT_TABLE table=test-table [id=c57a42539620488ba7c4ca2a7087ca31]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:00:31.983477 22316 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5dbc6f482ecd4c46a761422b1715dedc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:00:31.986529 22416 tablet_bootstrap.cc:492] T 5dbc6f482ecd4c46a761422b1715dedc P 3bf50bf4e77b45298676246bb4db1765: Bootstrap starting.
I20260811 19:00:31.988068 22417 tablet_bootstrap.cc:492] T 5dbc6f482ecd4c46a761422b1715dedc P 01dababe80fc41b48b6db09ba9cf2daa: Bootstrap starting.
I20260811 19:00:31.989221 22416 tablet_bootstrap.cc:654] T 5dbc6f482ecd4c46a761422b1715dedc P 3bf50bf4e77b45298676246bb4db1765: Neither blocks nor log segments found. Creating new log.
I20260811 19:00:31.990321 22416 log.cc:824] T 5dbc6f482ecd4c46a761422b1715dedc P 3bf50bf4e77b45298676246bb4db1765: Log is configured to *not* fsync() on all Append() calls
I20260811 19:00:31.990413 22417 tablet_bootstrap.cc:654] T 5dbc6f482ecd4c46a761422b1715dedc P 01dababe80fc41b48b6db09ba9cf2daa: Neither blocks nor log segments found. Creating new log.
I20260811 19:00:31.991143 22418 tablet_bootstrap.cc:492] T 5dbc6f482ecd4c46a761422b1715dedc P aa3ef1ab1dce4ff0876fed61e6719733: Bootstrap starting.
I20260811 19:00:31.991263 22417 log.cc:824] T 5dbc6f482ecd4c46a761422b1715dedc P 01dababe80fc41b48b6db09ba9cf2daa: Log is configured to *not* fsync() on all Append() calls
I20260811 19:00:31.993396 22417 tablet_bootstrap.cc:492] T 5dbc6f482ecd4c46a761422b1715dedc P 01dababe80fc41b48b6db09ba9cf2daa: No bootstrap required, opened a new log
I20260811 19:00:31.993557 22418 tablet_bootstrap.cc:654] T 5dbc6f482ecd4c46a761422b1715dedc P aa3ef1ab1dce4ff0876fed61e6719733: Neither blocks nor log segments found. Creating new log.
I20260811 19:00:31.993669 22417 ts_tablet_manager.cc:1397] T 5dbc6f482ecd4c46a761422b1715dedc P 01dababe80fc41b48b6db09ba9cf2daa: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 19:00:31.994006 22416 tablet_bootstrap.cc:492] T 5dbc6f482ecd4c46a761422b1715dedc P 3bf50bf4e77b45298676246bb4db1765: No bootstrap required, opened a new log
I20260811 19:00:31.994220 22416 ts_tablet_manager.cc:1397] T 5dbc6f482ecd4c46a761422b1715dedc P 3bf50bf4e77b45298676246bb4db1765: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260811 19:00:31.994474 22418 log.cc:824] T 5dbc6f482ecd4c46a761422b1715dedc P aa3ef1ab1dce4ff0876fed61e6719733: Log is configured to *not* fsync() on all Append() calls
I20260811 19:00:31.996656 22418 tablet_bootstrap.cc:492] T 5dbc6f482ecd4c46a761422b1715dedc P aa3ef1ab1dce4ff0876fed61e6719733: No bootstrap required, opened a new log
I20260811 19:00:31.996861 22418 ts_tablet_manager.cc:1397] T 5dbc6f482ecd4c46a761422b1715dedc P aa3ef1ab1dce4ff0876fed61e6719733: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.003s
I20260811 19:00:31.997218 22417 raft_consensus.cc:348] T 5dbc6f482ecd4c46a761422b1715dedc P 01dababe80fc41b48b6db09ba9cf2daa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bf50bf4e77b45298676246bb4db1765" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 36147 } } peers { permanent_uuid: "aa3ef1ab1dce4ff0876fed61e6719733" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 44129 } } peers { permanent_uuid: "01dababe80fc41b48b6db09ba9cf2daa" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34689 } }
I20260811 19:00:31.997514 22417 raft_consensus.cc:374] T 5dbc6f482ecd4c46a761422b1715dedc P 01dababe80fc41b48b6db09ba9cf2daa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:00:31.997570 22417 raft_consensus.cc:729] T 5dbc6f482ecd4c46a761422b1715dedc P 01dababe80fc41b48b6db09ba9cf2daa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01dababe80fc41b48b6db09ba9cf2daa, State: Initialized, Role: FOLLOWER
I20260811 19:00:31.998102 22417 consensus_queue.cc:260] T 5dbc6f482ecd4c46a761422b1715dedc P 01dababe80fc41b48b6db09ba9cf2daa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bf50bf4e77b45298676246bb4db1765" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 36147 } } peers { permanent_uuid: "aa3ef1ab1dce4ff0876fed61e6719733" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 44129 } } peers { permanent_uuid: "01dababe80fc41b48b6db09ba9cf2daa" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34689 } }
I20260811 19:00:31.998869 22416 raft_consensus.cc:348] T 5dbc6f482ecd4c46a761422b1715dedc P 3bf50bf4e77b45298676246bb4db1765 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bf50bf4e77b45298676246bb4db1765" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 36147 } } peers { permanent_uuid: "aa3ef1ab1dce4ff0876fed61e6719733" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 44129 } } peers { permanent_uuid: "01dababe80fc41b48b6db09ba9cf2daa" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34689 } }
I20260811 19:00:31.999157 22416 raft_consensus.cc:374] T 5dbc6f482ecd4c46a761422b1715dedc P 3bf50bf4e77b45298676246bb4db1765 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:00:31.999213 22416 raft_consensus.cc:729] T 5dbc6f482ecd4c46a761422b1715dedc P 3bf50bf4e77b45298676246bb4db1765 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bf50bf4e77b45298676246bb4db1765, State: Initialized, Role: FOLLOWER
I20260811 19:00:31.999234 22417 ts_tablet_manager.cc:1428] T 5dbc6f482ecd4c46a761422b1715dedc P 01dababe80fc41b48b6db09ba9cf2daa: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260811 19:00:31.999614 22231 heartbeater.cc:503] Master 127.13.109.62:37395 was elected leader, sending a full tablet report...
I20260811 19:00:31.999704 22416 consensus_queue.cc:260] T 5dbc6f482ecd4c46a761422b1715dedc P 3bf50bf4e77b45298676246bb4db1765 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bf50bf4e77b45298676246bb4db1765" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 36147 } } peers { permanent_uuid: "aa3ef1ab1dce4ff0876fed61e6719733" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 44129 } } peers { permanent_uuid: "01dababe80fc41b48b6db09ba9cf2daa" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34689 } }
I20260811 19:00:32.000451 22080 heartbeater.cc:503] Master 127.13.109.62:37395 was elected leader, sending a full tablet report...
I20260811 19:00:32.000772 22416 ts_tablet_manager.cc:1428] T 5dbc6f482ecd4c46a761422b1715dedc P 3bf50bf4e77b45298676246bb4db1765: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 19:00:32.001861 22418 raft_consensus.cc:348] T 5dbc6f482ecd4c46a761422b1715dedc P aa3ef1ab1dce4ff0876fed61e6719733 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bf50bf4e77b45298676246bb4db1765" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 36147 } } peers { permanent_uuid: "aa3ef1ab1dce4ff0876fed61e6719733" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 44129 } } peers { permanent_uuid: "01dababe80fc41b48b6db09ba9cf2daa" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34689 } }
I20260811 19:00:32.002164 22418 raft_consensus.cc:374] T 5dbc6f482ecd4c46a761422b1715dedc P aa3ef1ab1dce4ff0876fed61e6719733 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:00:32.002242 22418 raft_consensus.cc:729] T 5dbc6f482ecd4c46a761422b1715dedc P aa3ef1ab1dce4ff0876fed61e6719733 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa3ef1ab1dce4ff0876fed61e6719733, State: Initialized, Role: FOLLOWER
I20260811 19:00:32.002740 22418 consensus_queue.cc:260] T 5dbc6f482ecd4c46a761422b1715dedc P aa3ef1ab1dce4ff0876fed61e6719733 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bf50bf4e77b45298676246bb4db1765" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 36147 } } peers { permanent_uuid: "aa3ef1ab1dce4ff0876fed61e6719733" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 44129 } } peers { permanent_uuid: "01dababe80fc41b48b6db09ba9cf2daa" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34689 } }
I20260811 19:00:32.003866 22418 ts_tablet_manager.cc:1428] T 5dbc6f482ecd4c46a761422b1715dedc P aa3ef1ab1dce4ff0876fed61e6719733: Time spent starting tablet: real 0.007s user 0.007s sys 0.001s
I20260811 19:00:32.004352 22387 heartbeater.cc:503] Master 127.13.109.62:37395 was elected leader, sending a full tablet report...
I20260811 19:00:32.071609 22423 raft_consensus.cc:482] T 5dbc6f482ecd4c46a761422b1715dedc P 01dababe80fc41b48b6db09ba9cf2daa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:00:32.071906 22423 raft_consensus.cc:504] T 5dbc6f482ecd4c46a761422b1715dedc P 01dababe80fc41b48b6db09ba9cf2daa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bf50bf4e77b45298676246bb4db1765" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 36147 } } peers { permanent_uuid: "aa3ef1ab1dce4ff0876fed61e6719733" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 44129 } } peers { permanent_uuid: "01dababe80fc41b48b6db09ba9cf2daa" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34689 } }
I20260811 19:00:32.073185 22423 leader_election.cc:288] T 5dbc6f482ecd4c46a761422b1715dedc P 01dababe80fc41b48b6db09ba9cf2daa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3bf50bf4e77b45298676246bb4db1765 (127.13.109.1:36147), aa3ef1ab1dce4ff0876fed61e6719733 (127.13.109.3:44129)
I20260811 19:00:32.077708 22426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:32.073841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.3:44129 (local address 127.13.109.2:58845)
0811 19:00:32.074262 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:32.074283 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:32.074363 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:32.074590 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:32.074593 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:32.074610 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:00:32.074743 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:32.074750 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:32.075645 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:32.075648 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:32.076458 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:32.076468 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:32.077207 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:32.077211 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:32.077323 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:32.077367 (+ 44us) client_negotiation.cc:769] Sending connection context
0811 19:00:32.077441 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 19:00:32.077523 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":150,"threads_started":1}
I20260811 19:00:32.077719 22415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:32.073993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.2:58845 (local address 127.13.109.3:44129)
0811 19:00:32.074116 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:32.074121 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:32.074370 (+ 249us) server_negotiation.cc:407] Connection header received
0811 19:00:32.074433 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:32.074437 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:32.074494 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:32.074559 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:00:32.074871 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:32.075496 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:32.076603 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:32.077076 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:32.077173 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:32.077218 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:32.077474 (+ 256us) server_negotiation.cc:299] Negotiation successful
0811 19:00:32.077568 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 19:00:32.077895 22116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:32.073465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.1:36147 (local address 127.13.109.2:49285)
0811 19:00:32.073664 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:32.073683 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:32.073823 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:32.074181 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:32.074184 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:32.074208 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:00:32.074317 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:32.074343 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:32.075292 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:32.075296 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:32.076654 (+ 1358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:32.076667 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:32.077436 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:32.077442 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:32.077548 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:32.077570 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 19:00:32.077651 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 19:00:32.077720 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":82}
I20260811 19:00:32.077966 22411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:32.073598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.2:49285 (local address 127.13.109.1:36147)
0811 19:00:32.073752 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:32.073758 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:32.073775 (+ 17us) server_negotiation.cc:407] Connection header received
0811 19:00:32.073959 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:32.073962 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:32.074014 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:32.074136 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:00:32.074478 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:32.075164 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:32.076796 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:32.077289 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:32.077417 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:32.077471 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:32.077691 (+ 220us) server_negotiation.cc:299] Negotiation successful
0811 19:00:32.077815 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 19:00:32.078670 22337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5dbc6f482ecd4c46a761422b1715dedc" candidate_uuid: "01dababe80fc41b48b6db09ba9cf2daa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa3ef1ab1dce4ff0876fed61e6719733" is_pre_election: true
I20260811 19:00:32.078667 22031 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5dbc6f482ecd4c46a761422b1715dedc" candidate_uuid: "01dababe80fc41b48b6db09ba9cf2daa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bf50bf4e77b45298676246bb4db1765" is_pre_election: true
I20260811 19:00:32.079023 22031 raft_consensus.cc:2454] T 5dbc6f482ecd4c46a761422b1715dedc P 3bf50bf4e77b45298676246bb4db1765 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01dababe80fc41b48b6db09ba9cf2daa in term 0.
I20260811 19:00:32.079023 22337 raft_consensus.cc:2454] T 5dbc6f482ecd4c46a761422b1715dedc P aa3ef1ab1dce4ff0876fed61e6719733 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01dababe80fc41b48b6db09ba9cf2daa in term 0.
I20260811 19:00:32.079597 22108 leader_election.cc:302] T 5dbc6f482ecd4c46a761422b1715dedc P 01dababe80fc41b48b6db09ba9cf2daa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01dababe80fc41b48b6db09ba9cf2daa, aa3ef1ab1dce4ff0876fed61e6719733; no voters:
I20260811 19:00:32.079902 22423 raft_consensus.cc:2781] T 5dbc6f482ecd4c46a761422b1715dedc P 01dababe80fc41b48b6db09ba9cf2daa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 19:00:32.079998 22423 raft_consensus.cc:482] T 5dbc6f482ecd4c46a761422b1715dedc P 01dababe80fc41b48b6db09ba9cf2daa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 19:00:32.080050 22423 raft_consensus.cc:3037] T 5dbc6f482ecd4c46a761422b1715dedc P 01dababe80fc41b48b6db09ba9cf2daa [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:00:32.081275 22423 raft_consensus.cc:504] T 5dbc6f482ecd4c46a761422b1715dedc P 01dababe80fc41b48b6db09ba9cf2daa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bf50bf4e77b45298676246bb4db1765" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 36147 } } peers { permanent_uuid: "aa3ef1ab1dce4ff0876fed61e6719733" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 44129 } } peers { permanent_uuid: "01dababe80fc41b48b6db09ba9cf2daa" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34689 } }
I20260811 19:00:32.081645 22423 leader_election.cc:288] T 5dbc6f482ecd4c46a761422b1715dedc P 01dababe80fc41b48b6db09ba9cf2daa [CANDIDATE]: Term 1 election: Requested vote from peers 3bf50bf4e77b45298676246bb4db1765 (127.13.109.1:36147), aa3ef1ab1dce4ff0876fed61e6719733 (127.13.109.3:44129)
I20260811 19:00:32.082077 22031 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5dbc6f482ecd4c46a761422b1715dedc" candidate_uuid: "01dababe80fc41b48b6db09ba9cf2daa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bf50bf4e77b45298676246bb4db1765"
I20260811 19:00:32.082077 22337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5dbc6f482ecd4c46a761422b1715dedc" candidate_uuid: "01dababe80fc41b48b6db09ba9cf2daa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa3ef1ab1dce4ff0876fed61e6719733"
I20260811 19:00:32.082212 22031 raft_consensus.cc:3037] T 5dbc6f482ecd4c46a761422b1715dedc P 3bf50bf4e77b45298676246bb4db1765 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:00:32.082213 22337 raft_consensus.cc:3037] T 5dbc6f482ecd4c46a761422b1715dedc P aa3ef1ab1dce4ff0876fed61e6719733 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:00:32.083570 22337 raft_consensus.cc:2454] T 5dbc6f482ecd4c46a761422b1715dedc P aa3ef1ab1dce4ff0876fed61e6719733 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01dababe80fc41b48b6db09ba9cf2daa in term 1.
I20260811 19:00:32.083565 22031 raft_consensus.cc:2454] T 5dbc6f482ecd4c46a761422b1715dedc P 3bf50bf4e77b45298676246bb4db1765 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01dababe80fc41b48b6db09ba9cf2daa in term 1.
I20260811 19:00:32.083901 22108 leader_election.cc:302] T 5dbc6f482ecd4c46a761422b1715dedc P 01dababe80fc41b48b6db09ba9cf2daa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01dababe80fc41b48b6db09ba9cf2daa, 3bf50bf4e77b45298676246bb4db1765; no voters:
I20260811 19:00:32.084088 22423 raft_consensus.cc:2781] T 5dbc6f482ecd4c46a761422b1715dedc P 01dababe80fc41b48b6db09ba9cf2daa [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:00:32.084335 22423 raft_consensus.cc:686] T 5dbc6f482ecd4c46a761422b1715dedc P 01dababe80fc41b48b6db09ba9cf2daa [term 1 LEADER]: Becoming Leader. State: Replica: 01dababe80fc41b48b6db09ba9cf2daa, State: Running, Role: LEADER
I20260811 19:00:32.084686 22423 consensus_queue.cc:237] T 5dbc6f482ecd4c46a761422b1715dedc P 01dababe80fc41b48b6db09ba9cf2daa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bf50bf4e77b45298676246bb4db1765" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 36147 } } peers { permanent_uuid: "aa3ef1ab1dce4ff0876fed61e6719733" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 44129 } } peers { permanent_uuid: "01dababe80fc41b48b6db09ba9cf2daa" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34689 } }
I20260811 19:00:32.088160 21859 catalog_manager.cc:5495] T 5dbc6f482ecd4c46a761422b1715dedc P 01dababe80fc41b48b6db09ba9cf2daa reported cstate change: term changed from 0 to 1, leader changed from <none> to 01dababe80fc41b48b6db09ba9cf2daa (127.13.109.2). New cstate: current_term: 1 leader_uuid: "01dababe80fc41b48b6db09ba9cf2daa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bf50bf4e77b45298676246bb4db1765" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 36147 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa3ef1ab1dce4ff0876fed61e6719733" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 44129 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "01dababe80fc41b48b6db09ba9cf2daa" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34689 } health_report { overall_health: HEALTHY } } }
I20260811 19:00:32.099807 22414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:32.096182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55172 (local address 127.13.109.2:34689)
0811 19:00:32.096309 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:32.096312 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:32.096324 (+ 12us) server_negotiation.cc:407] Connection header received
0811 19:00:32.096389 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:32.096391 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:32.096440 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:32.096527 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:00:32.096838 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:32.097338 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:32.098310 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:32.098531 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:32.098610 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:32.098893 (+ 283us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:00:32.099397 (+ 504us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:00:32.099487 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:32.099628 (+ 141us) server_negotiation.cc:299] Negotiation successful
0811 19:00:32.099681 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 19:00:32.105836 22031 raft_consensus.cc:1261] T 5dbc6f482ecd4c46a761422b1715dedc P 3bf50bf4e77b45298676246bb4db1765 [term 1 FOLLOWER]: Refusing update from remote peer 01dababe80fc41b48b6db09ba9cf2daa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:00:32.105841 22337 raft_consensus.cc:1261] T 5dbc6f482ecd4c46a761422b1715dedc P aa3ef1ab1dce4ff0876fed61e6719733 [term 1 FOLLOWER]: Refusing update from remote peer 01dababe80fc41b48b6db09ba9cf2daa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:00:32.106614 22427 consensus_queue.cc:1035] T 5dbc6f482ecd4c46a761422b1715dedc P 01dababe80fc41b48b6db09ba9cf2daa [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bf50bf4e77b45298676246bb4db1765" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 36147 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:00:32.106827 22423 consensus_queue.cc:1035] T 5dbc6f482ecd4c46a761422b1715dedc P 01dababe80fc41b48b6db09ba9cf2daa [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa3ef1ab1dce4ff0876fed61e6719733" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 44129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:00:32.116976 22431 mvcc.cc:205] Tried to move back new op lower bound from 7317400912297828352 to 7317400912221679616. Current Snapshot: MvccSnapshot[applied={T|T < 7317400912297828352}]
I20260811 19:00:32.125177 22432 mvcc.cc:205] Tried to move back new op lower bound from 7317400912297828352 to 7317400912221679616. Current Snapshot: MvccSnapshot[applied={T|T < 7317400912297828352}]
I20260811 19:00:32.125214 21859 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60720:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:00:32.125392 21859 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60720:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:00:32.128827 21859 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 517db3110b44426982fea15cbe648df8: Sending DeleteTablet for 3 replicas of tablet 5dbc6f482ecd4c46a761422b1715dedc
I20260811 19:00:32.129859 22316 tablet_service.cc:1514] Processing DeleteTablet for tablet 5dbc6f482ecd4c46a761422b1715dedc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:00:32 UTC) from {username='oryx', principal='oryx/127.13.109.62@KRBTEST.COM'} at 127.0.0.1:59116
I20260811 19:00:32.129940 22161 tablet_service.cc:1514] Processing DeleteTablet for tablet 5dbc6f482ecd4c46a761422b1715dedc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:00:32 UTC) from {username='oryx', principal='oryx/127.13.109.62@KRBTEST.COM'} at 127.0.0.1:55168
I20260811 19:00:32.130613 22010 tablet_service.cc:1514] Processing DeleteTablet for tablet 5dbc6f482ecd4c46a761422b1715dedc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:00:32 UTC) from {username='oryx', principal='oryx/127.13.109.62@KRBTEST.COM'} at 127.0.0.1:39064
I20260811 19:00:32.130720 21859 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60720:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 19:00:32.130632 22009 tablet_service.cc:1514] Processing DeleteTablet for tablet 5dbc6f482ecd4c46a761422b1715dedc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:00:32 UTC) from {username='oryx', principal='oryx/127.13.109.62@KRBTEST.COM'} at 127.0.0.1:39064
W20260811 19:00:32.131170 21859 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 19:00:32.131377 22441 tablet_replica.cc:330] T 5dbc6f482ecd4c46a761422b1715dedc P aa3ef1ab1dce4ff0876fed61e6719733: stopping tablet replica
I20260811 19:00:32.131649 22441 raft_consensus.cc:2229] T 5dbc6f482ecd4c46a761422b1715dedc P aa3ef1ab1dce4ff0876fed61e6719733 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:00:32.131953 22441 raft_consensus.cc:2258] T 5dbc6f482ecd4c46a761422b1715dedc P aa3ef1ab1dce4ff0876fed61e6719733 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:00:32.132365 22442 tablet_replica.cc:330] T 5dbc6f482ecd4c46a761422b1715dedc P 01dababe80fc41b48b6db09ba9cf2daa: stopping tablet replica
I20260811 19:00:32.132597 22442 raft_consensus.cc:2229] T 5dbc6f482ecd4c46a761422b1715dedc P 01dababe80fc41b48b6db09ba9cf2daa [term 1 LEADER]: Raft consensus shutting down.
I20260811 19:00:32.132926 22442 raft_consensus.cc:2258] T 5dbc6f482ecd4c46a761422b1715dedc P 01dababe80fc41b48b6db09ba9cf2daa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:00:32.134196 22443 tablet_replica.cc:330] T 5dbc6f482ecd4c46a761422b1715dedc P 3bf50bf4e77b45298676246bb4db1765: stopping tablet replica
I20260811 19:00:32.134507 22443 raft_consensus.cc:2229] T 5dbc6f482ecd4c46a761422b1715dedc P 3bf50bf4e77b45298676246bb4db1765 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:00:32.134846 22443 raft_consensus.cc:2258] T 5dbc6f482ecd4c46a761422b1715dedc P 3bf50bf4e77b45298676246bb4db1765 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:00:32.135561 22441 ts_tablet_manager.cc:1905] T 5dbc6f482ecd4c46a761422b1715dedc P aa3ef1ab1dce4ff0876fed61e6719733: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:00:32.136289 22442 ts_tablet_manager.cc:1905] T 5dbc6f482ecd4c46a761422b1715dedc P 01dababe80fc41b48b6db09ba9cf2daa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:00:32.136874 22009 tablet_service.cc:1467] Processing CreateTablet for tablet bcbbfabbcbe34d13a117c9f13a031078 (DEFAULT_TABLE table=test-table [id=c802e2cec97e4b4fbcbfc19ca254fbed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:00:32.137272 22009 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bcbbfabbcbe34d13a117c9f13a031078. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:00:32.137632 22443 ts_tablet_manager.cc:1905] T 5dbc6f482ecd4c46a761422b1715dedc P 3bf50bf4e77b45298676246bb4db1765: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:00:32.139581 22442 ts_tablet_manager.cc:1918] T 5dbc6f482ecd4c46a761422b1715dedc P 01dababe80fc41b48b6db09ba9cf2daa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:00:32.139756 22442 log.cc:1196] T 5dbc6f482ecd4c46a761422b1715dedc P 01dababe80fc41b48b6db09ba9cf2daa: Deleting WAL directory at /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/minicluster-data/ts-1/wal/wals/5dbc6f482ecd4c46a761422b1715dedc
I20260811 19:00:32.140154 22416 tablet_bootstrap.cc:492] T bcbbfabbcbe34d13a117c9f13a031078 P 3bf50bf4e77b45298676246bb4db1765: Bootstrap starting.
I20260811 19:00:32.140158 22442 ts_tablet_manager.cc:1939] T 5dbc6f482ecd4c46a761422b1715dedc P 01dababe80fc41b48b6db09ba9cf2daa: Deleting consensus metadata
I20260811 19:00:32.140177 22443 ts_tablet_manager.cc:1918] T 5dbc6f482ecd4c46a761422b1715dedc P 3bf50bf4e77b45298676246bb4db1765: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:00:32.140285 22443 log.cc:1196] T 5dbc6f482ecd4c46a761422b1715dedc P 3bf50bf4e77b45298676246bb4db1765: Deleting WAL directory at /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/minicluster-data/ts-0/wal/wals/5dbc6f482ecd4c46a761422b1715dedc
I20260811 19:00:32.140688 22443 ts_tablet_manager.cc:1939] T 5dbc6f482ecd4c46a761422b1715dedc P 3bf50bf4e77b45298676246bb4db1765: Deleting consensus metadata
I20260811 19:00:32.140974 22316 tablet_service.cc:1467] Processing CreateTablet for tablet bcbbfabbcbe34d13a117c9f13a031078 (DEFAULT_TABLE table=test-table [id=c802e2cec97e4b4fbcbfc19ca254fbed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:00:32.141583 22416 tablet_bootstrap.cc:654] T bcbbfabbcbe34d13a117c9f13a031078 P 3bf50bf4e77b45298676246bb4db1765: Neither blocks nor log segments found. Creating new log.
I20260811 19:00:32.141673 22161 tablet_service.cc:1467] Processing CreateTablet for tablet bcbbfabbcbe34d13a117c9f13a031078 (DEFAULT_TABLE table=test-table [id=c802e2cec97e4b4fbcbfc19ca254fbed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:00:32.141953 22161 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bcbbfabbcbe34d13a117c9f13a031078. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:00:32.142848 22416 tablet_bootstrap.cc:492] T bcbbfabbcbe34d13a117c9f13a031078 P 3bf50bf4e77b45298676246bb4db1765: No bootstrap required, opened a new log
I20260811 19:00:32.142926 22416 ts_tablet_manager.cc:1397] T bcbbfabbcbe34d13a117c9f13a031078 P 3bf50bf4e77b45298676246bb4db1765: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260811 19:00:32.143385 22416 raft_consensus.cc:348] T bcbbfabbcbe34d13a117c9f13a031078 P 3bf50bf4e77b45298676246bb4db1765 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bf50bf4e77b45298676246bb4db1765" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 36147 } } peers { permanent_uuid: "01dababe80fc41b48b6db09ba9cf2daa" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34689 } } peers { permanent_uuid: "aa3ef1ab1dce4ff0876fed61e6719733" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 44129 } }
I20260811 19:00:32.143492 22416 raft_consensus.cc:374] T bcbbfabbcbe34d13a117c9f13a031078 P 3bf50bf4e77b45298676246bb4db1765 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:00:32.143524 22416 raft_consensus.cc:729] T bcbbfabbcbe34d13a117c9f13a031078 P 3bf50bf4e77b45298676246bb4db1765 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bf50bf4e77b45298676246bb4db1765, State: Initialized, Role: FOLLOWER
I20260811 19:00:32.143668 22417 tablet_bootstrap.cc:492] T bcbbfabbcbe34d13a117c9f13a031078 P 01dababe80fc41b48b6db09ba9cf2daa: Bootstrap starting.
I20260811 19:00:32.143793 22416 consensus_queue.cc:260] T bcbbfabbcbe34d13a117c9f13a031078 P 3bf50bf4e77b45298676246bb4db1765 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bf50bf4e77b45298676246bb4db1765" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 36147 } } peers { permanent_uuid: "01dababe80fc41b48b6db09ba9cf2daa" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34689 } } peers { permanent_uuid: "aa3ef1ab1dce4ff0876fed61e6719733" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 44129 } }
I20260811 19:00:32.144018 21845 catalog_manager.cc:4841] TS 3bf50bf4e77b45298676246bb4db1765 (127.13.109.1:36147): tablet 5dbc6f482ecd4c46a761422b1715dedc (table test-table [id=c57a42539620488ba7c4ca2a7087ca31]) successfully deleted
W20260811 19:00:32.144359 21845 catalog_manager.cc:4808] TS 3bf50bf4e77b45298676246bb4db1765 (127.13.109.1:36147): delete failed for tablet 5dbc6f482ecd4c46a761422b1715dedc because the tablet was not found. No further retry: Not found: Tablet not found: 5dbc6f482ecd4c46a761422b1715dedc
I20260811 19:00:32.144611 22416 ts_tablet_manager.cc:1428] T bcbbfabbcbe34d13a117c9f13a031078 P 3bf50bf4e77b45298676246bb4db1765: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260811 19:00:32.144591 21846 catalog_manager.cc:4841] TS 01dababe80fc41b48b6db09ba9cf2daa (127.13.109.2:34689): tablet 5dbc6f482ecd4c46a761422b1715dedc (table test-table [id=c57a42539620488ba7c4ca2a7087ca31]) successfully deleted
I20260811 19:00:32.144702 22417 tablet_bootstrap.cc:654] T bcbbfabbcbe34d13a117c9f13a031078 P 01dababe80fc41b48b6db09ba9cf2daa: Neither blocks nor log segments found. Creating new log.
I20260811 19:00:32.145574 22316 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bcbbfabbcbe34d13a117c9f13a031078. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:00:32.146080 22417 tablet_bootstrap.cc:492] T bcbbfabbcbe34d13a117c9f13a031078 P 01dababe80fc41b48b6db09ba9cf2daa: No bootstrap required, opened a new log
I20260811 19:00:32.146169 22417 ts_tablet_manager.cc:1397] T bcbbfabbcbe34d13a117c9f13a031078 P 01dababe80fc41b48b6db09ba9cf2daa: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 19:00:32.146595 22417 raft_consensus.cc:348] T bcbbfabbcbe34d13a117c9f13a031078 P 01dababe80fc41b48b6db09ba9cf2daa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bf50bf4e77b45298676246bb4db1765" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 36147 } } peers { permanent_uuid: "01dababe80fc41b48b6db09ba9cf2daa" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34689 } } peers { permanent_uuid: "aa3ef1ab1dce4ff0876fed61e6719733" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 44129 } }
I20260811 19:00:32.146719 22417 raft_consensus.cc:374] T bcbbfabbcbe34d13a117c9f13a031078 P 01dababe80fc41b48b6db09ba9cf2daa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:00:32.146768 22417 raft_consensus.cc:729] T bcbbfabbcbe34d13a117c9f13a031078 P 01dababe80fc41b48b6db09ba9cf2daa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01dababe80fc41b48b6db09ba9cf2daa, State: Initialized, Role: FOLLOWER
I20260811 19:00:32.146941 22417 consensus_queue.cc:260] T bcbbfabbcbe34d13a117c9f13a031078 P 01dababe80fc41b48b6db09ba9cf2daa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bf50bf4e77b45298676246bb4db1765" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 36147 } } peers { permanent_uuid: "01dababe80fc41b48b6db09ba9cf2daa" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34689 } } peers { permanent_uuid: "aa3ef1ab1dce4ff0876fed61e6719733" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 44129 } }
I20260811 19:00:32.147243 22417 ts_tablet_manager.cc:1428] T bcbbfabbcbe34d13a117c9f13a031078 P 01dababe80fc41b48b6db09ba9cf2daa: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 19:00:32.148257 22418 tablet_bootstrap.cc:492] T bcbbfabbcbe34d13a117c9f13a031078 P aa3ef1ab1dce4ff0876fed61e6719733: Bootstrap starting.
I20260811 19:00:32.149384 22418 tablet_bootstrap.cc:654] T bcbbfabbcbe34d13a117c9f13a031078 P aa3ef1ab1dce4ff0876fed61e6719733: Neither blocks nor log segments found. Creating new log.
I20260811 19:00:32.150826 22418 tablet_bootstrap.cc:492] T bcbbfabbcbe34d13a117c9f13a031078 P aa3ef1ab1dce4ff0876fed61e6719733: No bootstrap required, opened a new log
I20260811 19:00:32.150935 22418 ts_tablet_manager.cc:1397] T bcbbfabbcbe34d13a117c9f13a031078 P aa3ef1ab1dce4ff0876fed61e6719733: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 19:00:32.151413 22418 raft_consensus.cc:348] T bcbbfabbcbe34d13a117c9f13a031078 P aa3ef1ab1dce4ff0876fed61e6719733 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bf50bf4e77b45298676246bb4db1765" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 36147 } } peers { permanent_uuid: "01dababe80fc41b48b6db09ba9cf2daa" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34689 } } peers { permanent_uuid: "aa3ef1ab1dce4ff0876fed61e6719733" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 44129 } }
I20260811 19:00:32.151544 22418 raft_consensus.cc:374] T bcbbfabbcbe34d13a117c9f13a031078 P aa3ef1ab1dce4ff0876fed61e6719733 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:00:32.151588 22418 raft_consensus.cc:729] T bcbbfabbcbe34d13a117c9f13a031078 P aa3ef1ab1dce4ff0876fed61e6719733 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa3ef1ab1dce4ff0876fed61e6719733, State: Initialized, Role: FOLLOWER
I20260811 19:00:32.151731 22418 consensus_queue.cc:260] T bcbbfabbcbe34d13a117c9f13a031078 P aa3ef1ab1dce4ff0876fed61e6719733 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bf50bf4e77b45298676246bb4db1765" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 36147 } } peers { permanent_uuid: "01dababe80fc41b48b6db09ba9cf2daa" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34689 } } peers { permanent_uuid: "aa3ef1ab1dce4ff0876fed61e6719733" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 44129 } }
I20260811 19:00:32.152647 22418 ts_tablet_manager.cc:1428] T bcbbfabbcbe34d13a117c9f13a031078 P aa3ef1ab1dce4ff0876fed61e6719733: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
W20260811 19:00:32.152762 22232 tablet.cc:2369] T bcbbfabbcbe34d13a117c9f13a031078 P 01dababe80fc41b48b6db09ba9cf2daa: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 19:00:32.153733 22441 ts_tablet_manager.cc:1918] T 5dbc6f482ecd4c46a761422b1715dedc P aa3ef1ab1dce4ff0876fed61e6719733: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:00:32.153888 22441 log.cc:1196] T 5dbc6f482ecd4c46a761422b1715dedc P aa3ef1ab1dce4ff0876fed61e6719733: Deleting WAL directory at /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/minicluster-data/ts-2/wal/wals/5dbc6f482ecd4c46a761422b1715dedc
I20260811 19:00:32.154306 22441 ts_tablet_manager.cc:1939] T 5dbc6f482ecd4c46a761422b1715dedc P aa3ef1ab1dce4ff0876fed61e6719733: Deleting consensus metadata
I20260811 19:00:32.155599 21845 catalog_manager.cc:4841] TS aa3ef1ab1dce4ff0876fed61e6719733 (127.13.109.3:44129): tablet 5dbc6f482ecd4c46a761422b1715dedc (table test-table [id=c57a42539620488ba7c4ca2a7087ca31]) successfully deleted
W20260811 19:00:32.160799 22081 tablet.cc:2369] T bcbbfabbcbe34d13a117c9f13a031078 P 3bf50bf4e77b45298676246bb4db1765: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 19:00:32.170213 22439 raft_consensus.cc:482] T bcbbfabbcbe34d13a117c9f13a031078 P 01dababe80fc41b48b6db09ba9cf2daa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:00:32.170341 22439 raft_consensus.cc:504] T bcbbfabbcbe34d13a117c9f13a031078 P 01dababe80fc41b48b6db09ba9cf2daa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bf50bf4e77b45298676246bb4db1765" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 36147 } } peers { permanent_uuid: "01dababe80fc41b48b6db09ba9cf2daa" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34689 } } peers { permanent_uuid: "aa3ef1ab1dce4ff0876fed61e6719733" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 44129 } }
I20260811 19:00:32.170754 22439 leader_election.cc:288] T bcbbfabbcbe34d13a117c9f13a031078 P 01dababe80fc41b48b6db09ba9cf2daa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3bf50bf4e77b45298676246bb4db1765 (127.13.109.1:36147), aa3ef1ab1dce4ff0876fed61e6719733 (127.13.109.3:44129)
I20260811 19:00:32.171200 22031 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcbbfabbcbe34d13a117c9f13a031078" candidate_uuid: "01dababe80fc41b48b6db09ba9cf2daa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bf50bf4e77b45298676246bb4db1765" is_pre_election: true
I20260811 19:00:32.171206 22337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcbbfabbcbe34d13a117c9f13a031078" candidate_uuid: "01dababe80fc41b48b6db09ba9cf2daa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa3ef1ab1dce4ff0876fed61e6719733" is_pre_election: true
I20260811 19:00:32.171357 22337 raft_consensus.cc:2454] T bcbbfabbcbe34d13a117c9f13a031078 P aa3ef1ab1dce4ff0876fed61e6719733 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01dababe80fc41b48b6db09ba9cf2daa in term 0.
I20260811 19:00:32.171363 22031 raft_consensus.cc:2454] T bcbbfabbcbe34d13a117c9f13a031078 P 3bf50bf4e77b45298676246bb4db1765 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01dababe80fc41b48b6db09ba9cf2daa in term 0.
I20260811 19:00:32.171661 22108 leader_election.cc:302] T bcbbfabbcbe34d13a117c9f13a031078 P 01dababe80fc41b48b6db09ba9cf2daa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01dababe80fc41b48b6db09ba9cf2daa, 3bf50bf4e77b45298676246bb4db1765; no voters:
I20260811 19:00:32.171876 22439 raft_consensus.cc:2781] T bcbbfabbcbe34d13a117c9f13a031078 P 01dababe80fc41b48b6db09ba9cf2daa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 19:00:32.171952 22439 raft_consensus.cc:482] T bcbbfabbcbe34d13a117c9f13a031078 P 01dababe80fc41b48b6db09ba9cf2daa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 19:00:32.172004 22439 raft_consensus.cc:3037] T bcbbfabbcbe34d13a117c9f13a031078 P 01dababe80fc41b48b6db09ba9cf2daa [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:00:32.173017 22439 raft_consensus.cc:504] T bcbbfabbcbe34d13a117c9f13a031078 P 01dababe80fc41b48b6db09ba9cf2daa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bf50bf4e77b45298676246bb4db1765" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 36147 } } peers { permanent_uuid: "01dababe80fc41b48b6db09ba9cf2daa" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34689 } } peers { permanent_uuid: "aa3ef1ab1dce4ff0876fed61e6719733" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 44129 } }
I20260811 19:00:32.173350 22439 leader_election.cc:288] T bcbbfabbcbe34d13a117c9f13a031078 P 01dababe80fc41b48b6db09ba9cf2daa [CANDIDATE]: Term 1 election: Requested vote from peers 3bf50bf4e77b45298676246bb4db1765 (127.13.109.1:36147), aa3ef1ab1dce4ff0876fed61e6719733 (127.13.109.3:44129)
I20260811 19:00:32.173722 22031 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcbbfabbcbe34d13a117c9f13a031078" candidate_uuid: "01dababe80fc41b48b6db09ba9cf2daa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bf50bf4e77b45298676246bb4db1765"
I20260811 19:00:32.173717 22337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcbbfabbcbe34d13a117c9f13a031078" candidate_uuid: "01dababe80fc41b48b6db09ba9cf2daa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa3ef1ab1dce4ff0876fed61e6719733"
I20260811 19:00:32.173877 22031 raft_consensus.cc:3037] T bcbbfabbcbe34d13a117c9f13a031078 P 3bf50bf4e77b45298676246bb4db1765 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:00:32.173893 22337 raft_consensus.cc:3037] T bcbbfabbcbe34d13a117c9f13a031078 P aa3ef1ab1dce4ff0876fed61e6719733 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:00:32.174786 22337 raft_consensus.cc:2454] T bcbbfabbcbe34d13a117c9f13a031078 P aa3ef1ab1dce4ff0876fed61e6719733 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01dababe80fc41b48b6db09ba9cf2daa in term 1.
I20260811 19:00:32.175024 22031 raft_consensus.cc:2454] T bcbbfabbcbe34d13a117c9f13a031078 P 3bf50bf4e77b45298676246bb4db1765 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01dababe80fc41b48b6db09ba9cf2daa in term 1.
I20260811 19:00:32.175072 22108 leader_election.cc:302] T bcbbfabbcbe34d13a117c9f13a031078 P 01dababe80fc41b48b6db09ba9cf2daa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01dababe80fc41b48b6db09ba9cf2daa, aa3ef1ab1dce4ff0876fed61e6719733; no voters:
I20260811 19:00:32.175244 22439 raft_consensus.cc:2781] T bcbbfabbcbe34d13a117c9f13a031078 P 01dababe80fc41b48b6db09ba9cf2daa [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:00:32.175387 22439 raft_consensus.cc:686] T bcbbfabbcbe34d13a117c9f13a031078 P 01dababe80fc41b48b6db09ba9cf2daa [term 1 LEADER]: Becoming Leader. State: Replica: 01dababe80fc41b48b6db09ba9cf2daa, State: Running, Role: LEADER
I20260811 19:00:32.175576 22439 consensus_queue.cc:237] T bcbbfabbcbe34d13a117c9f13a031078 P 01dababe80fc41b48b6db09ba9cf2daa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bf50bf4e77b45298676246bb4db1765" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 36147 } } peers { permanent_uuid: "01dababe80fc41b48b6db09ba9cf2daa" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34689 } } peers { permanent_uuid: "aa3ef1ab1dce4ff0876fed61e6719733" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 44129 } }
I20260811 19:00:32.177320 21859 catalog_manager.cc:5495] T bcbbfabbcbe34d13a117c9f13a031078 P 01dababe80fc41b48b6db09ba9cf2daa reported cstate change: term changed from 0 to 1, leader changed from <none> to 01dababe80fc41b48b6db09ba9cf2daa (127.13.109.2). New cstate: current_term: 1 leader_uuid: "01dababe80fc41b48b6db09ba9cf2daa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bf50bf4e77b45298676246bb4db1765" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 36147 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "01dababe80fc41b48b6db09ba9cf2daa" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34689 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aa3ef1ab1dce4ff0876fed61e6719733" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 44129 } health_report { overall_health: UNKNOWN } } }
Aug 11 19:00:32 dist-test-slave-0kls krb5kdc[21811](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474831, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.109.62@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM
I20260811 19:00:32.216877 22453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:32.207785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:37395 (local address 127.0.0.1:60732)
0811 19:00:32.208168 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:32.208180 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:32.208329 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:32.208586 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:32.208589 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:32.209007 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:32.209106 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:32.209113 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:32.209897 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:32.209902 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:32.210453 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:32.210460 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:32.210914 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:32.210917 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:32.211013 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:32.211611 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:32.211623 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:32.213149 (+ 1526us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:00:32.215250 (+ 2101us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:32.215265 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:32.215271 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:32.215510 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:32.215811 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:32.215813 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:32.215814 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:32.215924 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:32.216377 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:00:32.216383 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:00:32.216500 (+ 117us) client_negotiation.cc:769] Sending connection context
0811 19:00:32.216606 (+ 106us) client_negotiation.cc:240] Negotiation successful
0811 19:00:32.216728 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"mutex_wait_us":1,"thread_start_us":115,"threads_started":1}
I20260811 19:00:32.216998 21913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:32.207856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60732 (local address 127.13.109.62:37395)
0811 19:00:32.207996 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:32.208000 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:32.208266 (+ 266us) server_negotiation.cc:407] Connection header received
0811 19:00:32.208427 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:32.208431 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:32.208477 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:32.208571 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:32.209224 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:32.209753 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:32.210587 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:32.210801 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:32.210906 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:32.213302 (+ 2396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:32.213306 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:32.213309 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:32.213337 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:32.215110 (+ 1773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:32.215617 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:32.215621 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:32.215623 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:32.215708 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:32.216010 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:32.216020 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:32.216022 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:32.216237 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:32.216353 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:32.216684 (+ 331us) server_negotiation.cc:299] Negotiation successful
0811 19:00:32.216818 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":56}
I20260811 19:00:32.221263 21859 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.13.109.62@KRBTEST.COM'} at 127.0.0.1:60732:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 19:00:32.222169 21859 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 19:00:32.227561 22161 tablet_service.cc:1467] Processing CreateTablet for tablet 29e7f225ac6e4e55b4a6aa921f8b9c23 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=951b717fd85d4f19be818872479f45c0]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 19:00:32.227581 22009 tablet_service.cc:1467] Processing CreateTablet for tablet 29e7f225ac6e4e55b4a6aa921f8b9c23 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=951b717fd85d4f19be818872479f45c0]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 19:00:32.227981 22161 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 29e7f225ac6e4e55b4a6aa921f8b9c23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:00:32.227981 22009 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 29e7f225ac6e4e55b4a6aa921f8b9c23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:00:32.231081 22416 tablet_bootstrap.cc:492] T 29e7f225ac6e4e55b4a6aa921f8b9c23 P 3bf50bf4e77b45298676246bb4db1765: Bootstrap starting.
I20260811 19:00:32.231256 22316 tablet_service.cc:1467] Processing CreateTablet for tablet 29e7f225ac6e4e55b4a6aa921f8b9c23 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=951b717fd85d4f19be818872479f45c0]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 19:00:32.231601 22316 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 29e7f225ac6e4e55b4a6aa921f8b9c23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:00:32.232101 22416 tablet_bootstrap.cc:654] T 29e7f225ac6e4e55b4a6aa921f8b9c23 P 3bf50bf4e77b45298676246bb4db1765: Neither blocks nor log segments found. Creating new log.
I20260811 19:00:32.234282 22418 tablet_bootstrap.cc:492] T 29e7f225ac6e4e55b4a6aa921f8b9c23 P aa3ef1ab1dce4ff0876fed61e6719733: Bootstrap starting.
I20260811 19:00:32.234474 22417 tablet_bootstrap.cc:492] T 29e7f225ac6e4e55b4a6aa921f8b9c23 P 01dababe80fc41b48b6db09ba9cf2daa: Bootstrap starting.
I20260811 19:00:32.235339 22418 tablet_bootstrap.cc:654] T 29e7f225ac6e4e55b4a6aa921f8b9c23 P aa3ef1ab1dce4ff0876fed61e6719733: Neither blocks nor log segments found. Creating new log.
I20260811 19:00:32.235448 22417 tablet_bootstrap.cc:654] T 29e7f225ac6e4e55b4a6aa921f8b9c23 P 01dababe80fc41b48b6db09ba9cf2daa: Neither blocks nor log segments found. Creating new log.
I20260811 19:00:32.237088 22418 tablet_bootstrap.cc:492] T 29e7f225ac6e4e55b4a6aa921f8b9c23 P aa3ef1ab1dce4ff0876fed61e6719733: No bootstrap required, opened a new log
I20260811 19:00:32.237185 22418 ts_tablet_manager.cc:1397] T 29e7f225ac6e4e55b4a6aa921f8b9c23 P aa3ef1ab1dce4ff0876fed61e6719733: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 19:00:32.237649 22416 tablet_bootstrap.cc:492] T 29e7f225ac6e4e55b4a6aa921f8b9c23 P 3bf50bf4e77b45298676246bb4db1765: No bootstrap required, opened a new log
I20260811 19:00:32.237663 22418 raft_consensus.cc:348] T 29e7f225ac6e4e55b4a6aa921f8b9c23 P aa3ef1ab1dce4ff0876fed61e6719733 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bf50bf4e77b45298676246bb4db1765" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 36147 } } peers { permanent_uuid: "01dababe80fc41b48b6db09ba9cf2daa" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34689 } } peers { permanent_uuid: "aa3ef1ab1dce4ff0876fed61e6719733" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 44129 } }
I20260811 19:00:32.237761 22416 ts_tablet_manager.cc:1397] T 29e7f225ac6e4e55b4a6aa921f8b9c23 P 3bf50bf4e77b45298676246bb4db1765: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.001s
I20260811 19:00:32.237815 22418 raft_consensus.cc:374] T 29e7f225ac6e4e55b4a6aa921f8b9c23 P aa3ef1ab1dce4ff0876fed61e6719733 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:00:32.237860 22418 raft_consensus.cc:729] T 29e7f225ac6e4e55b4a6aa921f8b9c23 P aa3ef1ab1dce4ff0876fed61e6719733 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa3ef1ab1dce4ff0876fed61e6719733, State: Initialized, Role: FOLLOWER
I20260811 19:00:32.237993 22418 consensus_queue.cc:260] T 29e7f225ac6e4e55b4a6aa921f8b9c23 P aa3ef1ab1dce4ff0876fed61e6719733 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bf50bf4e77b45298676246bb4db1765" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 36147 } } peers { permanent_uuid: "01dababe80fc41b48b6db09ba9cf2daa" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34689 } } peers { permanent_uuid: "aa3ef1ab1dce4ff0876fed61e6719733" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 44129 } }
I20260811 19:00:32.238276 22425 tablet_replica.cc:439] T 29e7f225ac6e4e55b4a6aa921f8b9c23 P aa3ef1ab1dce4ff0876fed61e6719733: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bf50bf4e77b45298676246bb4db1765" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 36147 } } peers { permanent_uuid: "01dababe80fc41b48b6db09ba9cf2daa" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34689 } } peers { permanent_uuid: "aa3ef1ab1dce4ff0876fed61e6719733" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 44129 } } }
I20260811 19:00:32.238300 22416 raft_consensus.cc:348] T 29e7f225ac6e4e55b4a6aa921f8b9c23 P 3bf50bf4e77b45298676246bb4db1765 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bf50bf4e77b45298676246bb4db1765" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 36147 } } peers { permanent_uuid: "01dababe80fc41b48b6db09ba9cf2daa" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34689 } } peers { permanent_uuid: "aa3ef1ab1dce4ff0876fed61e6719733" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 44129 } }
I20260811 19:00:32.238420 22425 tablet_replica.cc:442] T 29e7f225ac6e4e55b4a6aa921f8b9c23 P aa3ef1ab1dce4ff0876fed61e6719733: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 19:00:32.238423 22416 raft_consensus.cc:374] T 29e7f225ac6e4e55b4a6aa921f8b9c23 P 3bf50bf4e77b45298676246bb4db1765 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:00:32.238457 22416 raft_consensus.cc:729] T 29e7f225ac6e4e55b4a6aa921f8b9c23 P 3bf50bf4e77b45298676246bb4db1765 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bf50bf4e77b45298676246bb4db1765, State: Initialized, Role: FOLLOWER
I20260811 19:00:32.238576 22416 consensus_queue.cc:260] T 29e7f225ac6e4e55b4a6aa921f8b9c23 P 3bf50bf4e77b45298676246bb4db1765 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bf50bf4e77b45298676246bb4db1765" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 36147 } } peers { permanent_uuid: "01dababe80fc41b48b6db09ba9cf2daa" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34689 } } peers { permanent_uuid: "aa3ef1ab1dce4ff0876fed61e6719733" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 44129 } }
I20260811 19:00:32.238655 22418 ts_tablet_manager.cc:1428] T 29e7f225ac6e4e55b4a6aa921f8b9c23 P aa3ef1ab1dce4ff0876fed61e6719733: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260811 19:00:32.238889 22416 ts_tablet_manager.cc:1428] T 29e7f225ac6e4e55b4a6aa921f8b9c23 P 3bf50bf4e77b45298676246bb4db1765: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 19:00:32.238828 22424 tablet_replica.cc:439] T 29e7f225ac6e4e55b4a6aa921f8b9c23 P 3bf50bf4e77b45298676246bb4db1765: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bf50bf4e77b45298676246bb4db1765" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 36147 } } peers { permanent_uuid: "01dababe80fc41b48b6db09ba9cf2daa" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34689 } } peers { permanent_uuid: "aa3ef1ab1dce4ff0876fed61e6719733" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 44129 } } }
I20260811 19:00:32.238967 22424 tablet_replica.cc:442] T 29e7f225ac6e4e55b4a6aa921f8b9c23 P 3bf50bf4e77b45298676246bb4db1765: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 19:00:32.242738 22417 tablet_bootstrap.cc:492] T 29e7f225ac6e4e55b4a6aa921f8b9c23 P 01dababe80fc41b48b6db09ba9cf2daa: No bootstrap required, opened a new log
I20260811 19:00:32.242823 22417 ts_tablet_manager.cc:1397] T 29e7f225ac6e4e55b4a6aa921f8b9c23 P 01dababe80fc41b48b6db09ba9cf2daa: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20260811 19:00:32.243263 22417 raft_consensus.cc:348] T 29e7f225ac6e4e55b4a6aa921f8b9c23 P 01dababe80fc41b48b6db09ba9cf2daa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bf50bf4e77b45298676246bb4db1765" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 36147 } } peers { permanent_uuid: "01dababe80fc41b48b6db09ba9cf2daa" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34689 } } peers { permanent_uuid: "aa3ef1ab1dce4ff0876fed61e6719733" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 44129 } }
I20260811 19:00:32.243369 22417 raft_consensus.cc:374] T 29e7f225ac6e4e55b4a6aa921f8b9c23 P 01dababe80fc41b48b6db09ba9cf2daa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:00:32.243392 22417 raft_consensus.cc:729] T 29e7f225ac6e4e55b4a6aa921f8b9c23 P 01dababe80fc41b48b6db09ba9cf2daa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01dababe80fc41b48b6db09ba9cf2daa, State: Initialized, Role: FOLLOWER
I20260811 19:00:32.243587 22417 consensus_queue.cc:260] T 29e7f225ac6e4e55b4a6aa921f8b9c23 P 01dababe80fc41b48b6db09ba9cf2daa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bf50bf4e77b45298676246bb4db1765" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 36147 } } peers { permanent_uuid: "01dababe80fc41b48b6db09ba9cf2daa" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34689 } } peers { permanent_uuid: "aa3ef1ab1dce4ff0876fed61e6719733" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 44129 } }
I20260811 19:00:32.243880 22417 ts_tablet_manager.cc:1428] T 29e7f225ac6e4e55b4a6aa921f8b9c23 P 01dababe80fc41b48b6db09ba9cf2daa: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 19:00:32.243825 22439 tablet_replica.cc:439] T 29e7f225ac6e4e55b4a6aa921f8b9c23 P 01dababe80fc41b48b6db09ba9cf2daa: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bf50bf4e77b45298676246bb4db1765" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 36147 } } peers { permanent_uuid: "01dababe80fc41b48b6db09ba9cf2daa" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34689 } } peers { permanent_uuid: "aa3ef1ab1dce4ff0876fed61e6719733" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 44129 } } }
I20260811 19:00:32.243961 22439 tablet_replica.cc:442] T 29e7f225ac6e4e55b4a6aa921f8b9c23 P 01dababe80fc41b48b6db09ba9cf2daa: This TxnStatusTablet replica's current role is: FOLLOWER
W20260811 19:00:32.395247 22388 tablet.cc:2369] T 29e7f225ac6e4e55b4a6aa921f8b9c23 P aa3ef1ab1dce4ff0876fed61e6719733: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 19:00:32.531113 22439 raft_consensus.cc:482] T 29e7f225ac6e4e55b4a6aa921f8b9c23 P 01dababe80fc41b48b6db09ba9cf2daa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:00:32.531286 22439 raft_consensus.cc:504] T 29e7f225ac6e4e55b4a6aa921f8b9c23 P 01dababe80fc41b48b6db09ba9cf2daa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bf50bf4e77b45298676246bb4db1765" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 36147 } } peers { permanent_uuid: "01dababe80fc41b48b6db09ba9cf2daa" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34689 } } peers { permanent_uuid: "aa3ef1ab1dce4ff0876fed61e6719733" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 44129 } }
I20260811 19:00:32.531795 22439 leader_election.cc:288] T 29e7f225ac6e4e55b4a6aa921f8b9c23 P 01dababe80fc41b48b6db09ba9cf2daa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3bf50bf4e77b45298676246bb4db1765 (127.13.109.1:36147), aa3ef1ab1dce4ff0876fed61e6719733 (127.13.109.3:44129)
I20260811 19:00:32.532409 22031 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29e7f225ac6e4e55b4a6aa921f8b9c23" candidate_uuid: "01dababe80fc41b48b6db09ba9cf2daa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bf50bf4e77b45298676246bb4db1765" is_pre_election: true
I20260811 19:00:32.532487 22337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29e7f225ac6e4e55b4a6aa921f8b9c23" candidate_uuid: "01dababe80fc41b48b6db09ba9cf2daa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa3ef1ab1dce4ff0876fed61e6719733" is_pre_election: true
I20260811 19:00:32.532583 22031 raft_consensus.cc:2454] T 29e7f225ac6e4e55b4a6aa921f8b9c23 P 3bf50bf4e77b45298676246bb4db1765 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01dababe80fc41b48b6db09ba9cf2daa in term 0.
I20260811 19:00:32.532698 22337 raft_consensus.cc:2454] T 29e7f225ac6e4e55b4a6aa921f8b9c23 P aa3ef1ab1dce4ff0876fed61e6719733 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01dababe80fc41b48b6db09ba9cf2daa in term 0.
I20260811 19:00:32.532965 22108 leader_election.cc:302] T 29e7f225ac6e4e55b4a6aa921f8b9c23 P 01dababe80fc41b48b6db09ba9cf2daa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01dababe80fc41b48b6db09ba9cf2daa, 3bf50bf4e77b45298676246bb4db1765; no voters:
I20260811 19:00:32.533238 22439 raft_consensus.cc:2781] T 29e7f225ac6e4e55b4a6aa921f8b9c23 P 01dababe80fc41b48b6db09ba9cf2daa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 19:00:32.533303 22439 raft_consensus.cc:482] T 29e7f225ac6e4e55b4a6aa921f8b9c23 P 01dababe80fc41b48b6db09ba9cf2daa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 19:00:32.533330 22439 raft_consensus.cc:3037] T 29e7f225ac6e4e55b4a6aa921f8b9c23 P 01dababe80fc41b48b6db09ba9cf2daa [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:00:32.534440 22439 raft_consensus.cc:504] T 29e7f225ac6e4e55b4a6aa921f8b9c23 P 01dababe80fc41b48b6db09ba9cf2daa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bf50bf4e77b45298676246bb4db1765" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 36147 } } peers { permanent_uuid: "01dababe80fc41b48b6db09ba9cf2daa" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34689 } } peers { permanent_uuid: "aa3ef1ab1dce4ff0876fed61e6719733" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 44129 } }
I20260811 19:00:32.534785 22439 leader_election.cc:288] T 29e7f225ac6e4e55b4a6aa921f8b9c23 P 01dababe80fc41b48b6db09ba9cf2daa [CANDIDATE]: Term 1 election: Requested vote from peers 3bf50bf4e77b45298676246bb4db1765 (127.13.109.1:36147), aa3ef1ab1dce4ff0876fed61e6719733 (127.13.109.3:44129)
I20260811 19:00:32.535168 22031 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29e7f225ac6e4e55b4a6aa921f8b9c23" candidate_uuid: "01dababe80fc41b48b6db09ba9cf2daa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bf50bf4e77b45298676246bb4db1765"
I20260811 19:00:32.535245 22337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29e7f225ac6e4e55b4a6aa921f8b9c23" candidate_uuid: "01dababe80fc41b48b6db09ba9cf2daa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa3ef1ab1dce4ff0876fed61e6719733"
I20260811 19:00:32.535318 22031 raft_consensus.cc:3037] T 29e7f225ac6e4e55b4a6aa921f8b9c23 P 3bf50bf4e77b45298676246bb4db1765 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:00:32.535418 22337 raft_consensus.cc:3037] T 29e7f225ac6e4e55b4a6aa921f8b9c23 P aa3ef1ab1dce4ff0876fed61e6719733 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:00:32.536439 22031 raft_consensus.cc:2454] T 29e7f225ac6e4e55b4a6aa921f8b9c23 P 3bf50bf4e77b45298676246bb4db1765 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01dababe80fc41b48b6db09ba9cf2daa in term 1.
I20260811 19:00:32.536504 22337 raft_consensus.cc:2454] T 29e7f225ac6e4e55b4a6aa921f8b9c23 P aa3ef1ab1dce4ff0876fed61e6719733 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01dababe80fc41b48b6db09ba9cf2daa in term 1.
I20260811 19:00:32.536849 22108 leader_election.cc:302] T 29e7f225ac6e4e55b4a6aa921f8b9c23 P 01dababe80fc41b48b6db09ba9cf2daa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01dababe80fc41b48b6db09ba9cf2daa, aa3ef1ab1dce4ff0876fed61e6719733; no voters:
I20260811 19:00:32.537022 22439 raft_consensus.cc:2781] T 29e7f225ac6e4e55b4a6aa921f8b9c23 P 01dababe80fc41b48b6db09ba9cf2daa [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:00:32.537151 22439 raft_consensus.cc:686] T 29e7f225ac6e4e55b4a6aa921f8b9c23 P 01dababe80fc41b48b6db09ba9cf2daa [term 1 LEADER]: Becoming Leader. State: Replica: 01dababe80fc41b48b6db09ba9cf2daa, State: Running, Role: LEADER
I20260811 19:00:32.537328 22439 consensus_queue.cc:237] T 29e7f225ac6e4e55b4a6aa921f8b9c23 P 01dababe80fc41b48b6db09ba9cf2daa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bf50bf4e77b45298676246bb4db1765" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 36147 } } peers { permanent_uuid: "01dababe80fc41b48b6db09ba9cf2daa" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34689 } } peers { permanent_uuid: "aa3ef1ab1dce4ff0876fed61e6719733" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 44129 } }
I20260811 19:00:32.538182 22427 tablet_replica.cc:439] T 29e7f225ac6e4e55b4a6aa921f8b9c23 P 01dababe80fc41b48b6db09ba9cf2daa: TxnStatusTablet state changed. Reason: New leader 01dababe80fc41b48b6db09ba9cf2daa. Latest consensus state: current_term: 1 leader_uuid: "01dababe80fc41b48b6db09ba9cf2daa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bf50bf4e77b45298676246bb4db1765" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 36147 } } peers { permanent_uuid: "01dababe80fc41b48b6db09ba9cf2daa" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34689 } } peers { permanent_uuid: "aa3ef1ab1dce4ff0876fed61e6719733" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 44129 } } }
I20260811 19:00:32.538295 22427 tablet_replica.cc:442] T 29e7f225ac6e4e55b4a6aa921f8b9c23 P 01dababe80fc41b48b6db09ba9cf2daa: This TxnStatusTablet replica's current role is: LEADER
I20260811 19:00:32.538542 22461 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 19:00:32.538731 22461 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 19:00:32.539333 21859 catalog_manager.cc:5495] T 29e7f225ac6e4e55b4a6aa921f8b9c23 P 01dababe80fc41b48b6db09ba9cf2daa reported cstate change: term changed from 0 to 1, leader changed from <none> to 01dababe80fc41b48b6db09ba9cf2daa (127.13.109.2). New cstate: current_term: 1 leader_uuid: "01dababe80fc41b48b6db09ba9cf2daa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bf50bf4e77b45298676246bb4db1765" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 36147 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "01dababe80fc41b48b6db09ba9cf2daa" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34689 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aa3ef1ab1dce4ff0876fed61e6719733" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 44129 } health_report { overall_health: UNKNOWN } } }
I20260811 19:00:32.600631 22414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:32.596965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55184 (local address 127.13.109.2:34689)
0811 19:00:32.597113 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:32.597117 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:32.597132 (+ 15us) server_negotiation.cc:407] Connection header received
0811 19:00:32.597241 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:32.597243 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:32.597290 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:32.597357 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:00:32.597650 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:32.598291 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:32.599551 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:32.599783 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:32.599898 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:32.600210 (+ 312us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:00:32.600287 (+ 77us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:00:32.600347 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:32.600462 (+ 115us) server_negotiation.cc:299] Negotiation successful
0811 19:00:32.600510 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 19:00:32.600705 22453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:32.596744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.2:34689 (local address 127.0.0.1:55184)
0811 19:00:32.597004 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:32.597020 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:32.597123 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:32.597402 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:32.597405 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:32.597413 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 19:00:32.597536 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:32.597543 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:32.598402 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:32.598406 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:32.599432 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:32.599443 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:32.599895 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:32.599899 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:32.599990 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:32.600052 (+ 62us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 19:00:32.600375 (+ 323us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 19:00:32.600391 (+ 16us) client_negotiation.cc:769] Sending connection context
0811 19:00:32.600458 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 19:00:32.600558 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156}
I20260811 19:00:32.603823 22031 raft_consensus.cc:1261] T 29e7f225ac6e4e55b4a6aa921f8b9c23 P 3bf50bf4e77b45298676246bb4db1765 [term 1 FOLLOWER]: Refusing update from remote peer 01dababe80fc41b48b6db09ba9cf2daa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:00:32.603794 22337 raft_consensus.cc:1261] T 29e7f225ac6e4e55b4a6aa921f8b9c23 P aa3ef1ab1dce4ff0876fed61e6719733 [term 1 FOLLOWER]: Refusing update from remote peer 01dababe80fc41b48b6db09ba9cf2daa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:00:32.604301 22427 consensus_queue.cc:1035] T 29e7f225ac6e4e55b4a6aa921f8b9c23 P 01dababe80fc41b48b6db09ba9cf2daa [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bf50bf4e77b45298676246bb4db1765" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 36147 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:00:32.604468 22439 consensus_queue.cc:1035] T 29e7f225ac6e4e55b4a6aa921f8b9c23 P 01dababe80fc41b48b6db09ba9cf2daa [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa3ef1ab1dce4ff0876fed61e6719733" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 44129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:00:32.606022 22425 tablet_replica.cc:439] T 29e7f225ac6e4e55b4a6aa921f8b9c23 P aa3ef1ab1dce4ff0876fed61e6719733: TxnStatusTablet state changed. Reason: New leader 01dababe80fc41b48b6db09ba9cf2daa. Latest consensus state: current_term: 1 leader_uuid: "01dababe80fc41b48b6db09ba9cf2daa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bf50bf4e77b45298676246bb4db1765" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 36147 } } peers { permanent_uuid: "01dababe80fc41b48b6db09ba9cf2daa" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34689 } } peers { permanent_uuid: "aa3ef1ab1dce4ff0876fed61e6719733" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 44129 } } }
I20260811 19:00:32.606165 22425 tablet_replica.cc:442] T 29e7f225ac6e4e55b4a6aa921f8b9c23 P aa3ef1ab1dce4ff0876fed61e6719733: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 19:00:32.606410 22424 tablet_replica.cc:439] T 29e7f225ac6e4e55b4a6aa921f8b9c23 P 3bf50bf4e77b45298676246bb4db1765: TxnStatusTablet state changed. Reason: New leader 01dababe80fc41b48b6db09ba9cf2daa. Latest consensus state: current_term: 1 leader_uuid: "01dababe80fc41b48b6db09ba9cf2daa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bf50bf4e77b45298676246bb4db1765" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 36147 } } peers { permanent_uuid: "01dababe80fc41b48b6db09ba9cf2daa" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34689 } } peers { permanent_uuid: "aa3ef1ab1dce4ff0876fed61e6719733" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 44129 } } }
I20260811 19:00:32.606539 22424 tablet_replica.cc:442] T 29e7f225ac6e4e55b4a6aa921f8b9c23 P 3bf50bf4e77b45298676246bb4db1765: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 19:00:32.608006 22427 tablet_replica.cc:439] T 29e7f225ac6e4e55b4a6aa921f8b9c23 P 01dababe80fc41b48b6db09ba9cf2daa: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "01dababe80fc41b48b6db09ba9cf2daa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bf50bf4e77b45298676246bb4db1765" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 36147 } } peers { permanent_uuid: "01dababe80fc41b48b6db09ba9cf2daa" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34689 } } peers { permanent_uuid: "aa3ef1ab1dce4ff0876fed61e6719733" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 44129 } } }
I20260811 19:00:32.608121 22427 tablet_replica.cc:442] T 29e7f225ac6e4e55b4a6aa921f8b9c23 P 01dababe80fc41b48b6db09ba9cf2daa: This TxnStatusTablet replica's current role is: LEADER
I20260811 19:00:32.608618 22430 mvcc.cc:205] Tried to move back new op lower bound from 7317400914340061184 to 7317400914074341376. Current Snapshot: MvccSnapshot[applied={T|T < 7317400914340061184 or (T in {7317400914340061184})}]
I20260811 19:00:32.609575 22424 tablet_replica.cc:439] T 29e7f225ac6e4e55b4a6aa921f8b9c23 P 3bf50bf4e77b45298676246bb4db1765: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "01dababe80fc41b48b6db09ba9cf2daa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bf50bf4e77b45298676246bb4db1765" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 36147 } } peers { permanent_uuid: "01dababe80fc41b48b6db09ba9cf2daa" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34689 } } peers { permanent_uuid: "aa3ef1ab1dce4ff0876fed61e6719733" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 44129 } } }
I20260811 19:00:32.609685 22424 tablet_replica.cc:442] T 29e7f225ac6e4e55b4a6aa921f8b9c23 P 3bf50bf4e77b45298676246bb4db1765: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 19:00:32.610035 22439 tablet_replica.cc:439] T 29e7f225ac6e4e55b4a6aa921f8b9c23 P 01dababe80fc41b48b6db09ba9cf2daa: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "01dababe80fc41b48b6db09ba9cf2daa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bf50bf4e77b45298676246bb4db1765" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 36147 } } peers { permanent_uuid: "01dababe80fc41b48b6db09ba9cf2daa" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34689 } } peers { permanent_uuid: "aa3ef1ab1dce4ff0876fed61e6719733" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 44129 } } }
I20260811 19:00:32.610144 22439 tablet_replica.cc:442] T 29e7f225ac6e4e55b4a6aa921f8b9c23 P 01dababe80fc41b48b6db09ba9cf2daa: This TxnStatusTablet replica's current role is: LEADER
I20260811 19:00:32.610937 22425 tablet_replica.cc:439] T 29e7f225ac6e4e55b4a6aa921f8b9c23 P aa3ef1ab1dce4ff0876fed61e6719733: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "01dababe80fc41b48b6db09ba9cf2daa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bf50bf4e77b45298676246bb4db1765" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 36147 } } peers { permanent_uuid: "01dababe80fc41b48b6db09ba9cf2daa" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34689 } } peers { permanent_uuid: "aa3ef1ab1dce4ff0876fed61e6719733" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 44129 } } }
I20260811 19:00:32.611052 22425 tablet_replica.cc:442] T 29e7f225ac6e4e55b4a6aa921f8b9c23 P aa3ef1ab1dce4ff0876fed61e6719733: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 19:00:32.623306 22414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:32.619643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.2:56979 (local address 127.13.109.2:34689)
0811 19:00:32.619774 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:32.619778 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:32.620039 (+ 261us) server_negotiation.cc:407] Connection header received
0811 19:00:32.620175 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:32.620177 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:32.620217 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:32.620278 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:00:32.620550 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:32.621044 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:32.622259 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:32.622461 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:32.622540 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:32.622845 (+ 305us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:00:32.622909 (+ 64us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:00:32.622962 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:32.623127 (+ 165us) server_negotiation.cc:299] Negotiation successful
0811 19:00:32.623173 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":43}
I20260811 19:00:32.623329 22468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:32.619571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.2:34689 (local address 127.13.109.2:56979)
0811 19:00:32.619954 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:32.619969 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:32.620080 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:32.620310 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:32.620313 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:32.620320 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 19:00:32.620453 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:32.620459 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:32.621144 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:32.621147 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:32.622145 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:32.622156 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:32.622580 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:32.622583 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:32.622667 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:32.622719 (+ 52us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 19:00:32.622994 (+ 275us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 19:00:32.623000 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 19:00:32.623070 (+ 70us) client_negotiation.cc:240] Negotiation successful
0811 19:00:32.623150 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"mutex_wait_us":17,"thread_start_us":108,"threads_started":1}
I20260811 19:00:32.631135 22337 raft_consensus.cc:1261] T bcbbfabbcbe34d13a117c9f13a031078 P aa3ef1ab1dce4ff0876fed61e6719733 [term 1 FOLLOWER]: Refusing update from remote peer 01dababe80fc41b48b6db09ba9cf2daa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:00:32.631143 22031 raft_consensus.cc:1261] T bcbbfabbcbe34d13a117c9f13a031078 P 3bf50bf4e77b45298676246bb4db1765 [term 1 FOLLOWER]: Refusing update from remote peer 01dababe80fc41b48b6db09ba9cf2daa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:00:32.631531 22423 consensus_queue.cc:1035] T bcbbfabbcbe34d13a117c9f13a031078 P 01dababe80fc41b48b6db09ba9cf2daa [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bf50bf4e77b45298676246bb4db1765" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 36147 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:00:32.631723 22427 consensus_queue.cc:1035] T bcbbfabbcbe34d13a117c9f13a031078 P 01dababe80fc41b48b6db09ba9cf2daa [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa3ef1ab1dce4ff0876fed61e6719733" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 44129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:00:32.673462 21858 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60720:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:00:32.673643 21858 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60720:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:00:32.675874 21858 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 517db3110b44426982fea15cbe648df8: Sending DeleteTablet for 3 replicas of tablet bcbbfabbcbe34d13a117c9f13a031078
I20260811 19:00:32.676548 22316 tablet_service.cc:1514] Processing DeleteTablet for tablet bcbbfabbcbe34d13a117c9f13a031078 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:00:32 UTC) from {username='oryx', principal='oryx/127.13.109.62@KRBTEST.COM'} at 127.0.0.1:59116
I20260811 19:00:32.676836 22009 tablet_service.cc:1514] Processing DeleteTablet for tablet bcbbfabbcbe34d13a117c9f13a031078 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:00:32 UTC) from {username='oryx', principal='oryx/127.13.109.62@KRBTEST.COM'} at 127.0.0.1:39064
I20260811 19:00:32.677392 22473 tablet_replica.cc:330] T bcbbfabbcbe34d13a117c9f13a031078 P 3bf50bf4e77b45298676246bb4db1765: stopping tablet replica
I20260811 19:00:32.677609 22473 raft_consensus.cc:2229] T bcbbfabbcbe34d13a117c9f13a031078 P 3bf50bf4e77b45298676246bb4db1765 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:00:32.677842 22473 raft_consensus.cc:2258] T bcbbfabbcbe34d13a117c9f13a031078 P 3bf50bf4e77b45298676246bb4db1765 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:00:32.678701 22473 ts_tablet_manager.cc:1905] T bcbbfabbcbe34d13a117c9f13a031078 P 3bf50bf4e77b45298676246bb4db1765: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:00:32.680938 22161 tablet_service.cc:1514] Processing DeleteTablet for tablet bcbbfabbcbe34d13a117c9f13a031078 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:00:32 UTC) from {username='oryx', principal='oryx/127.13.109.62@KRBTEST.COM'} at 127.0.0.1:55168
I20260811 19:00:32.682771 22473 ts_tablet_manager.cc:1918] T bcbbfabbcbe34d13a117c9f13a031078 P 3bf50bf4e77b45298676246bb4db1765: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 19:00:32.682859 22473 log.cc:1196] T bcbbfabbcbe34d13a117c9f13a031078 P 3bf50bf4e77b45298676246bb4db1765: Deleting WAL directory at /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/minicluster-data/ts-0/wal/wals/bcbbfabbcbe34d13a117c9f13a031078
I20260811 19:00:32.683122 22473 ts_tablet_manager.cc:1939] T bcbbfabbcbe34d13a117c9f13a031078 P 3bf50bf4e77b45298676246bb4db1765: Deleting consensus metadata
I20260811 19:00:32.683929 21845 catalog_manager.cc:4841] TS 3bf50bf4e77b45298676246bb4db1765 (127.13.109.1:36147): tablet bcbbfabbcbe34d13a117c9f13a031078 (table test-table [id=c802e2cec97e4b4fbcbfc19ca254fbed]) successfully deleted
I20260811 19:00:32.685997 22472 tablet_replica.cc:330] T bcbbfabbcbe34d13a117c9f13a031078 P aa3ef1ab1dce4ff0876fed61e6719733: stopping tablet replica
I20260811 19:00:32.686192 22472 raft_consensus.cc:2229] T bcbbfabbcbe34d13a117c9f13a031078 P aa3ef1ab1dce4ff0876fed61e6719733 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:00:32.686347 22472 raft_consensus.cc:2258] T bcbbfabbcbe34d13a117c9f13a031078 P aa3ef1ab1dce4ff0876fed61e6719733 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:00:32.687110 22472 ts_tablet_manager.cc:1905] T bcbbfabbcbe34d13a117c9f13a031078 P aa3ef1ab1dce4ff0876fed61e6719733: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:00:32.689378 22476 tablet_replica.cc:330] T bcbbfabbcbe34d13a117c9f13a031078 P 01dababe80fc41b48b6db09ba9cf2daa: stopping tablet replica
I20260811 19:00:32.689533 22476 raft_consensus.cc:2229] T bcbbfabbcbe34d13a117c9f13a031078 P 01dababe80fc41b48b6db09ba9cf2daa [term 1 LEADER]: Raft consensus shutting down.
I20260811 19:00:32.689749 22476 raft_consensus.cc:2258] T bcbbfabbcbe34d13a117c9f13a031078 P 01dababe80fc41b48b6db09ba9cf2daa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:00:32.690192 22472 ts_tablet_manager.cc:1918] T bcbbfabbcbe34d13a117c9f13a031078 P aa3ef1ab1dce4ff0876fed61e6719733: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 19:00:32.690290 22472 log.cc:1196] T bcbbfabbcbe34d13a117c9f13a031078 P aa3ef1ab1dce4ff0876fed61e6719733: Deleting WAL directory at /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/minicluster-data/ts-2/wal/wals/bcbbfabbcbe34d13a117c9f13a031078
I20260811 19:00:32.690441 22476 ts_tablet_manager.cc:1905] T bcbbfabbcbe34d13a117c9f13a031078 P 01dababe80fc41b48b6db09ba9cf2daa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:00:32.690584 22472 ts_tablet_manager.cc:1939] T bcbbfabbcbe34d13a117c9f13a031078 P aa3ef1ab1dce4ff0876fed61e6719733: Deleting consensus metadata
I20260811 19:00:32.691458 21845 catalog_manager.cc:4841] TS aa3ef1ab1dce4ff0876fed61e6719733 (127.13.109.3:44129): tablet bcbbfabbcbe34d13a117c9f13a031078 (table test-table [id=c802e2cec97e4b4fbcbfc19ca254fbed]) successfully deleted
I20260811 19:00:32.693413 22476 ts_tablet_manager.cc:1918] T bcbbfabbcbe34d13a117c9f13a031078 P 01dababe80fc41b48b6db09ba9cf2daa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 19:00:32.693498 22476 log.cc:1196] T bcbbfabbcbe34d13a117c9f13a031078 P 01dababe80fc41b48b6db09ba9cf2daa: Deleting WAL directory at /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/minicluster-data/ts-1/wal/wals/bcbbfabbcbe34d13a117c9f13a031078
I20260811 19:00:32.693763 22476 ts_tablet_manager.cc:1939] T bcbbfabbcbe34d13a117c9f13a031078 P 01dababe80fc41b48b6db09ba9cf2daa: Deleting consensus metadata
I20260811 19:00:32.694542 21846 catalog_manager.cc:4841] TS 01dababe80fc41b48b6db09ba9cf2daa (127.13.109.2:34689): tablet bcbbfabbcbe34d13a117c9f13a031078 (table test-table [id=c802e2cec97e4b4fbcbfc19ca254fbed]) successfully deleted
Aug 11 19:00:32 dist-test-slave-0kls krb5kdc[21811](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474832, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 19:00:32 dist-test-slave-0kls krb5kdc[21811](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:00:32 dist-test-slave-0kls krb5kdc[21811](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
W20260811 19:00:32.709290 21913 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:00:32.702749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60742 (local address 127.13.109.62:37395)
0811 19:00:32.702898 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:32.702904 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:32.703197 (+ 293us) server_negotiation.cc:407] Connection header received
0811 19:00:32.703370 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:32.703373 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:32.703421 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:32.703507 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:32.704263 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:32.704801 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:32.705679 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:32.706006 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:32.706123 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:32.709147 (+ 3024us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60742: BlockingRecv error: recv got EOF from 127.0.0.1:60742 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 11 19:00:32 dist-test-slave-0kls krb5kdc[21811](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474832, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM
I20260811 19:00:32.719044 21913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:32.711128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60754 (local address 127.13.109.62:37395)
0811 19:00:32.711307 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:32.711310 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:32.711388 (+ 78us) server_negotiation.cc:407] Connection header received
0811 19:00:32.711472 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:32.711474 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:32.711521 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:32.711590 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:32.712166 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:32.712722 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:32.713597 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:32.713926 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:32.714015 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:32.716376 (+ 2361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:32.716379 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:32.716381 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:32.716404 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:32.717636 (+ 1232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:32.718076 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:32.718079 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:32.718080 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:32.718163 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:32.718388 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:32.718390 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:32.718391 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:32.718550 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:32.718647 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:32.718830 (+ 183us) server_negotiation.cc:299] Negotiation successful
0811 19:00:32.718922 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260811 19:00:32.721652 21858 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60754:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 19:00:32.722127 21858 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 19:00:32.726962 22009 tablet_service.cc:1467] Processing CreateTablet for tablet 42730b2693ef456fb43e171d6b210556 (DEFAULT_TABLE table=test-table [id=522b14a9544448a3b0f71a5d05997124]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:00:32.727123 22161 tablet_service.cc:1467] Processing CreateTablet for tablet 42730b2693ef456fb43e171d6b210556 (DEFAULT_TABLE table=test-table [id=522b14a9544448a3b0f71a5d05997124]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:00:32.727183 22316 tablet_service.cc:1467] Processing CreateTablet for tablet 42730b2693ef456fb43e171d6b210556 (DEFAULT_TABLE table=test-table [id=522b14a9544448a3b0f71a5d05997124]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:00:32.727350 22009 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42730b2693ef456fb43e171d6b210556. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:00:32.727427 22316 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42730b2693ef456fb43e171d6b210556. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:00:32.727427 22161 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42730b2693ef456fb43e171d6b210556. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:00:32.729413 22416 tablet_bootstrap.cc:492] T 42730b2693ef456fb43e171d6b210556 P 3bf50bf4e77b45298676246bb4db1765: Bootstrap starting.
I20260811 19:00:32.729875 22418 tablet_bootstrap.cc:492] T 42730b2693ef456fb43e171d6b210556 P aa3ef1ab1dce4ff0876fed61e6719733: Bootstrap starting.
I20260811 19:00:32.730002 22417 tablet_bootstrap.cc:492] T 42730b2693ef456fb43e171d6b210556 P 01dababe80fc41b48b6db09ba9cf2daa: Bootstrap starting.
I20260811 19:00:32.730446 22416 tablet_bootstrap.cc:654] T 42730b2693ef456fb43e171d6b210556 P 3bf50bf4e77b45298676246bb4db1765: Neither blocks nor log segments found. Creating new log.
I20260811 19:00:32.730665 22417 tablet_bootstrap.cc:654] T 42730b2693ef456fb43e171d6b210556 P 01dababe80fc41b48b6db09ba9cf2daa: Neither blocks nor log segments found. Creating new log.
I20260811 19:00:32.730861 22418 tablet_bootstrap.cc:654] T 42730b2693ef456fb43e171d6b210556 P aa3ef1ab1dce4ff0876fed61e6719733: Neither blocks nor log segments found. Creating new log.
I20260811 19:00:32.731966 22417 tablet_bootstrap.cc:492] T 42730b2693ef456fb43e171d6b210556 P 01dababe80fc41b48b6db09ba9cf2daa: No bootstrap required, opened a new log
I20260811 19:00:32.732018 22418 tablet_bootstrap.cc:492] T 42730b2693ef456fb43e171d6b210556 P aa3ef1ab1dce4ff0876fed61e6719733: No bootstrap required, opened a new log
I20260811 19:00:32.732065 22417 ts_tablet_manager.cc:1397] T 42730b2693ef456fb43e171d6b210556 P 01dababe80fc41b48b6db09ba9cf2daa: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 19:00:32.732102 22418 ts_tablet_manager.cc:1397] T 42730b2693ef456fb43e171d6b210556 P aa3ef1ab1dce4ff0876fed61e6719733: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260811 19:00:32.732496 22417 raft_consensus.cc:348] T 42730b2693ef456fb43e171d6b210556 P 01dababe80fc41b48b6db09ba9cf2daa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bf50bf4e77b45298676246bb4db1765" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 36147 } } peers { permanent_uuid: "01dababe80fc41b48b6db09ba9cf2daa" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34689 } } peers { permanent_uuid: "aa3ef1ab1dce4ff0876fed61e6719733" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 44129 } }
I20260811 19:00:32.732496 22418 raft_consensus.cc:348] T 42730b2693ef456fb43e171d6b210556 P aa3ef1ab1dce4ff0876fed61e6719733 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bf50bf4e77b45298676246bb4db1765" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 36147 } } peers { permanent_uuid: "01dababe80fc41b48b6db09ba9cf2daa" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34689 } } peers { permanent_uuid: "aa3ef1ab1dce4ff0876fed61e6719733" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 44129 } }
I20260811 19:00:32.732638 22418 raft_consensus.cc:374] T 42730b2693ef456fb43e171d6b210556 P aa3ef1ab1dce4ff0876fed61e6719733 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:00:32.732638 22417 raft_consensus.cc:374] T 42730b2693ef456fb43e171d6b210556 P 01dababe80fc41b48b6db09ba9cf2daa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:00:32.732677 22418 raft_consensus.cc:729] T 42730b2693ef456fb43e171d6b210556 P aa3ef1ab1dce4ff0876fed61e6719733 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa3ef1ab1dce4ff0876fed61e6719733, State: Initialized, Role: FOLLOWER
I20260811 19:00:32.732677 22417 raft_consensus.cc:729] T 42730b2693ef456fb43e171d6b210556 P 01dababe80fc41b48b6db09ba9cf2daa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01dababe80fc41b48b6db09ba9cf2daa, State: Initialized, Role: FOLLOWER
I20260811 19:00:32.732820 22418 consensus_queue.cc:260] T 42730b2693ef456fb43e171d6b210556 P aa3ef1ab1dce4ff0876fed61e6719733 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bf50bf4e77b45298676246bb4db1765" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 36147 } } peers { permanent_uuid: "01dababe80fc41b48b6db09ba9cf2daa" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34689 } } peers { permanent_uuid: "aa3ef1ab1dce4ff0876fed61e6719733" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 44129 } }
I20260811 19:00:32.732817 22417 consensus_queue.cc:260] T 42730b2693ef456fb43e171d6b210556 P 01dababe80fc41b48b6db09ba9cf2daa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bf50bf4e77b45298676246bb4db1765" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 36147 } } peers { permanent_uuid: "01dababe80fc41b48b6db09ba9cf2daa" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34689 } } peers { permanent_uuid: "aa3ef1ab1dce4ff0876fed61e6719733" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 44129 } }
I20260811 19:00:32.733098 22417 ts_tablet_manager.cc:1428] T 42730b2693ef456fb43e171d6b210556 P 01dababe80fc41b48b6db09ba9cf2daa: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 19:00:32.733125 22418 ts_tablet_manager.cc:1428] T 42730b2693ef456fb43e171d6b210556 P aa3ef1ab1dce4ff0876fed61e6719733: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 19:00:32.734264 22416 tablet_bootstrap.cc:492] T 42730b2693ef456fb43e171d6b210556 P 3bf50bf4e77b45298676246bb4db1765: No bootstrap required, opened a new log
I20260811 19:00:32.734354 22416 ts_tablet_manager.cc:1397] T 42730b2693ef456fb43e171d6b210556 P 3bf50bf4e77b45298676246bb4db1765: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260811 19:00:32.734900 22416 raft_consensus.cc:348] T 42730b2693ef456fb43e171d6b210556 P 3bf50bf4e77b45298676246bb4db1765 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bf50bf4e77b45298676246bb4db1765" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 36147 } } peers { permanent_uuid: "01dababe80fc41b48b6db09ba9cf2daa" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34689 } } peers { permanent_uuid: "aa3ef1ab1dce4ff0876fed61e6719733" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 44129 } }
I20260811 19:00:32.735049 22416 raft_consensus.cc:374] T 42730b2693ef456fb43e171d6b210556 P 3bf50bf4e77b45298676246bb4db1765 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:00:32.735085 22416 raft_consensus.cc:729] T 42730b2693ef456fb43e171d6b210556 P 3bf50bf4e77b45298676246bb4db1765 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bf50bf4e77b45298676246bb4db1765, State: Initialized, Role: FOLLOWER
I20260811 19:00:32.735239 22416 consensus_queue.cc:260] T 42730b2693ef456fb43e171d6b210556 P 3bf50bf4e77b45298676246bb4db1765 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bf50bf4e77b45298676246bb4db1765" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 36147 } } peers { permanent_uuid: "01dababe80fc41b48b6db09ba9cf2daa" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34689 } } peers { permanent_uuid: "aa3ef1ab1dce4ff0876fed61e6719733" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 44129 } }
I20260811 19:00:32.735591 22416 ts_tablet_manager.cc:1428] T 42730b2693ef456fb43e171d6b210556 P 3bf50bf4e77b45298676246bb4db1765: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 19:00:32.935763 22425 raft_consensus.cc:482] T 42730b2693ef456fb43e171d6b210556 P aa3ef1ab1dce4ff0876fed61e6719733 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:00:32.935940 22425 raft_consensus.cc:504] T 42730b2693ef456fb43e171d6b210556 P aa3ef1ab1dce4ff0876fed61e6719733 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bf50bf4e77b45298676246bb4db1765" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 36147 } } peers { permanent_uuid: "01dababe80fc41b48b6db09ba9cf2daa" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34689 } } peers { permanent_uuid: "aa3ef1ab1dce4ff0876fed61e6719733" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 44129 } }
I20260811 19:00:32.937301 22425 leader_election.cc:288] T 42730b2693ef456fb43e171d6b210556 P aa3ef1ab1dce4ff0876fed61e6719733 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3bf50bf4e77b45298676246bb4db1765 (127.13.109.1:36147), 01dababe80fc41b48b6db09ba9cf2daa (127.13.109.2:34689)
I20260811 19:00:32.941681 22492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:32.937629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.2:34689 (local address 127.13.109.3:35103)
0811 19:00:32.938040 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:32.938058 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:32.938196 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:32.938580 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:32.938583 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:32.938601 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:00:32.938720 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:32.938726 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:32.939575 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:32.939581 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:32.940440 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:32.940450 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:32.941179 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:32.941186 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:32.941278 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:32.941329 (+ 51us) client_negotiation.cc:769] Sending connection context
0811 19:00:32.941410 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 19:00:32.941471 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"mutex_wait_us":2,"thread_start_us":119,"threads_started":1}
I20260811 19:00:32.941726 22414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:32.937757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.3:35103 (local address 127.13.109.2:34689)
0811 19:00:32.937934 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:32.937939 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:32.938152 (+ 213us) server_negotiation.cc:407] Connection header received
0811 19:00:32.938366 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:32.938372 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:32.938443 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:32.938518 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:00:32.938856 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:32.939402 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:32.940592 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:32.941037 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:32.941148 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:32.941246 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:32.941448 (+ 202us) server_negotiation.cc:299] Negotiation successful
0811 19:00:32.941567 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260811 19:00:32.941895 22493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:32.937633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.1:36147 (local address 127.13.109.3:35489)
0811 19:00:32.938178 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:32.938191 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:32.938320 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:32.938624 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:32.938628 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:32.938662 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:00:32.938758 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:32.938764 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:32.939607 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:32.939610 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:32.940467 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:32.940492 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:32.941250 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:32.941256 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:32.941429 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:32.941459 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 19:00:32.941558 (+ 99us) client_negotiation.cc:240] Negotiation successful
0811 19:00:32.941631 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":462,"mutex_wait_us":170,"spinlock_wait_cycles":896,"thread_start_us":100,"threads_started":1}
I20260811 19:00:32.942008 22494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:32.937705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.3:35489 (local address 127.13.109.1:36147)
0811 19:00:32.938099 (+ 394us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:32.938106 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:32.938273 (+ 167us) server_negotiation.cc:407] Connection header received
0811 19:00:32.938430 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:32.938433 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:32.938476 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:32.938588 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:00:32.938919 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:32.939448 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:32.940646 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:32.941109 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:32.941255 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:32.941315 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:32.941588 (+ 273us) server_negotiation.cc:299] Negotiation successful
0811 19:00:32.941667 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":130,"threads_started":1}
I20260811 19:00:32.942348 22182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42730b2693ef456fb43e171d6b210556" candidate_uuid: "aa3ef1ab1dce4ff0876fed61e6719733" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01dababe80fc41b48b6db09ba9cf2daa" is_pre_election: true
I20260811 19:00:32.942592 22182 raft_consensus.cc:2454] T 42730b2693ef456fb43e171d6b210556 P 01dababe80fc41b48b6db09ba9cf2daa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa3ef1ab1dce4ff0876fed61e6719733 in term 0.
I20260811 19:00:32.942914 22030 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42730b2693ef456fb43e171d6b210556" candidate_uuid: "aa3ef1ab1dce4ff0876fed61e6719733" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bf50bf4e77b45298676246bb4db1765" is_pre_election: true
I20260811 19:00:32.943066 22030 raft_consensus.cc:2454] T 42730b2693ef456fb43e171d6b210556 P 3bf50bf4e77b45298676246bb4db1765 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa3ef1ab1dce4ff0876fed61e6719733 in term 0.
I20260811 19:00:32.943081 22262 leader_election.cc:302] T 42730b2693ef456fb43e171d6b210556 P aa3ef1ab1dce4ff0876fed61e6719733 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01dababe80fc41b48b6db09ba9cf2daa, aa3ef1ab1dce4ff0876fed61e6719733; no voters:
I20260811 19:00:32.943431 22425 raft_consensus.cc:2781] T 42730b2693ef456fb43e171d6b210556 P aa3ef1ab1dce4ff0876fed61e6719733 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 19:00:32.943508 22425 raft_consensus.cc:482] T 42730b2693ef456fb43e171d6b210556 P aa3ef1ab1dce4ff0876fed61e6719733 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 19:00:32.943560 22425 raft_consensus.cc:3037] T 42730b2693ef456fb43e171d6b210556 P aa3ef1ab1dce4ff0876fed61e6719733 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:00:32.944614 22425 raft_consensus.cc:504] T 42730b2693ef456fb43e171d6b210556 P aa3ef1ab1dce4ff0876fed61e6719733 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bf50bf4e77b45298676246bb4db1765" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 36147 } } peers { permanent_uuid: "01dababe80fc41b48b6db09ba9cf2daa" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34689 } } peers { permanent_uuid: "aa3ef1ab1dce4ff0876fed61e6719733" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 44129 } }
I20260811 19:00:32.944989 22425 leader_election.cc:288] T 42730b2693ef456fb43e171d6b210556 P aa3ef1ab1dce4ff0876fed61e6719733 [CANDIDATE]: Term 1 election: Requested vote from peers 3bf50bf4e77b45298676246bb4db1765 (127.13.109.1:36147), 01dababe80fc41b48b6db09ba9cf2daa (127.13.109.2:34689)
I20260811 19:00:32.945465 22182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42730b2693ef456fb43e171d6b210556" candidate_uuid: "aa3ef1ab1dce4ff0876fed61e6719733" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01dababe80fc41b48b6db09ba9cf2daa"
I20260811 19:00:32.945590 22182 raft_consensus.cc:3037] T 42730b2693ef456fb43e171d6b210556 P 01dababe80fc41b48b6db09ba9cf2daa [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:00:32.945617 22030 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42730b2693ef456fb43e171d6b210556" candidate_uuid: "aa3ef1ab1dce4ff0876fed61e6719733" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bf50bf4e77b45298676246bb4db1765"
I20260811 19:00:32.945747 22030 raft_consensus.cc:3037] T 42730b2693ef456fb43e171d6b210556 P 3bf50bf4e77b45298676246bb4db1765 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:00:32.946700 22182 raft_consensus.cc:2454] T 42730b2693ef456fb43e171d6b210556 P 01dababe80fc41b48b6db09ba9cf2daa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa3ef1ab1dce4ff0876fed61e6719733 in term 1.
I20260811 19:00:32.946790 22030 raft_consensus.cc:2454] T 42730b2693ef456fb43e171d6b210556 P 3bf50bf4e77b45298676246bb4db1765 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa3ef1ab1dce4ff0876fed61e6719733 in term 1.
I20260811 19:00:32.946998 22262 leader_election.cc:302] T 42730b2693ef456fb43e171d6b210556 P aa3ef1ab1dce4ff0876fed61e6719733 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01dababe80fc41b48b6db09ba9cf2daa, aa3ef1ab1dce4ff0876fed61e6719733; no voters:
I20260811 19:00:32.947227 22425 raft_consensus.cc:2781] T 42730b2693ef456fb43e171d6b210556 P aa3ef1ab1dce4ff0876fed61e6719733 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:00:32.947389 22425 raft_consensus.cc:686] T 42730b2693ef456fb43e171d6b210556 P aa3ef1ab1dce4ff0876fed61e6719733 [term 1 LEADER]: Becoming Leader. State: Replica: aa3ef1ab1dce4ff0876fed61e6719733, State: Running, Role: LEADER
I20260811 19:00:32.947661 22425 consensus_queue.cc:237] T 42730b2693ef456fb43e171d6b210556 P aa3ef1ab1dce4ff0876fed61e6719733 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bf50bf4e77b45298676246bb4db1765" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 36147 } } peers { permanent_uuid: "01dababe80fc41b48b6db09ba9cf2daa" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34689 } } peers { permanent_uuid: "aa3ef1ab1dce4ff0876fed61e6719733" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 44129 } }
I20260811 19:00:32.950220 21860 catalog_manager.cc:5495] T 42730b2693ef456fb43e171d6b210556 P aa3ef1ab1dce4ff0876fed61e6719733 reported cstate change: term changed from 0 to 1, leader changed from <none> to aa3ef1ab1dce4ff0876fed61e6719733 (127.13.109.3). New cstate: current_term: 1 leader_uuid: "aa3ef1ab1dce4ff0876fed61e6719733" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bf50bf4e77b45298676246bb4db1765" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 36147 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "01dababe80fc41b48b6db09ba9cf2daa" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34689 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa3ef1ab1dce4ff0876fed61e6719733" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 44129 } health_report { overall_health: HEALTHY } } }
I20260811 19:00:32.975116 22498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:32.971116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59124 (local address 127.13.109.3:44129)
0811 19:00:32.971432 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:32.971457 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:32.971472 (+ 15us) server_negotiation.cc:407] Connection header received
0811 19:00:32.971564 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:32.971566 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:32.971626 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:32.971748 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:00:32.972014 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:32.972623 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:32.973471 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:32.973719 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:32.973881 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:32.974086 (+ 205us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:00:32.974579 (+ 493us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:00:32.974678 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:32.974814 (+ 136us) server_negotiation.cc:299] Negotiation successful
0811 19:00:32.974867 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":226,"thread_start_us":124,"threads_started":1}
I20260811 19:00:32.978179 22030 raft_consensus.cc:1261] T 42730b2693ef456fb43e171d6b210556 P 3bf50bf4e77b45298676246bb4db1765 [term 1 FOLLOWER]: Refusing update from remote peer aa3ef1ab1dce4ff0876fed61e6719733: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:00:32.978359 22182 raft_consensus.cc:1261] T 42730b2693ef456fb43e171d6b210556 P 01dababe80fc41b48b6db09ba9cf2daa [term 1 FOLLOWER]: Refusing update from remote peer aa3ef1ab1dce4ff0876fed61e6719733: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:00:32.978832 22495 consensus_queue.cc:1035] T 42730b2693ef456fb43e171d6b210556 P aa3ef1ab1dce4ff0876fed61e6719733 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bf50bf4e77b45298676246bb4db1765" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 36147 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:00:32.978995 22425 consensus_queue.cc:1035] T 42730b2693ef456fb43e171d6b210556 P aa3ef1ab1dce4ff0876fed61e6719733 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01dababe80fc41b48b6db09ba9cf2daa" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34689 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:00:32.991139 21858 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60754:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:00:32.991331 21858 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60754:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:00:32.993538 21858 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 517db3110b44426982fea15cbe648df8: Sending DeleteTablet for 3 replicas of tablet 42730b2693ef456fb43e171d6b210556
I20260811 19:00:32.994377 22161 tablet_service.cc:1514] Processing DeleteTablet for tablet 42730b2693ef456fb43e171d6b210556 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:00:32 UTC) from {username='oryx', principal='oryx/127.13.109.62@KRBTEST.COM'} at 127.0.0.1:55168
I20260811 19:00:32.994451 22009 tablet_service.cc:1514] Processing DeleteTablet for tablet 42730b2693ef456fb43e171d6b210556 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:00:32 UTC) from {username='oryx', principal='oryx/127.13.109.62@KRBTEST.COM'} at 127.0.0.1:39064
I20260811 19:00:32.994680 22473 tablet_replica.cc:330] T 42730b2693ef456fb43e171d6b210556 P 3bf50bf4e77b45298676246bb4db1765: stopping tablet replica
I20260811 19:00:32.994688 22476 tablet_replica.cc:330] T 42730b2693ef456fb43e171d6b210556 P 01dababe80fc41b48b6db09ba9cf2daa: stopping tablet replica
I20260811 19:00:32.994829 22476 raft_consensus.cc:2229] T 42730b2693ef456fb43e171d6b210556 P 01dababe80fc41b48b6db09ba9cf2daa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:00:32.994835 22473 raft_consensus.cc:2229] T 42730b2693ef456fb43e171d6b210556 P 3bf50bf4e77b45298676246bb4db1765 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:00:32.994964 22476 raft_consensus.cc:2258] T 42730b2693ef456fb43e171d6b210556 P 01dababe80fc41b48b6db09ba9cf2daa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:00:32.994992 22473 raft_consensus.cc:2258] T 42730b2693ef456fb43e171d6b210556 P 3bf50bf4e77b45298676246bb4db1765 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:00:32.995091 21858 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60754:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 19:00:32.995565 21858 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 19:00:32.995826 22316 tablet_service.cc:1514] Processing DeleteTablet for tablet 42730b2693ef456fb43e171d6b210556 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:00:32 UTC) from {username='oryx', principal='oryx/127.13.109.62@KRBTEST.COM'} at 127.0.0.1:59116
I20260811 19:00:32.995939 22473 ts_tablet_manager.cc:1905] T 42730b2693ef456fb43e171d6b210556 P 3bf50bf4e77b45298676246bb4db1765: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:00:32.996016 22472 tablet_replica.cc:330] T 42730b2693ef456fb43e171d6b210556 P aa3ef1ab1dce4ff0876fed61e6719733: stopping tablet replica
I20260811 19:00:32.996124 22472 raft_consensus.cc:2229] T 42730b2693ef456fb43e171d6b210556 P aa3ef1ab1dce4ff0876fed61e6719733 [term 1 LEADER]: Raft consensus shutting down.
I20260811 19:00:32.996439 22472 raft_consensus.cc:2258] T 42730b2693ef456fb43e171d6b210556 P aa3ef1ab1dce4ff0876fed61e6719733 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:00:32.997390 22476 ts_tablet_manager.cc:1905] T 42730b2693ef456fb43e171d6b210556 P 01dababe80fc41b48b6db09ba9cf2daa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:00:32.997442 22472 ts_tablet_manager.cc:1905] T 42730b2693ef456fb43e171d6b210556 P aa3ef1ab1dce4ff0876fed61e6719733: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:00:32.998867 22473 ts_tablet_manager.cc:1918] T 42730b2693ef456fb43e171d6b210556 P 3bf50bf4e77b45298676246bb4db1765: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:00:32.998968 22473 log.cc:1196] T 42730b2693ef456fb43e171d6b210556 P 3bf50bf4e77b45298676246bb4db1765: Deleting WAL directory at /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/minicluster-data/ts-0/wal/wals/42730b2693ef456fb43e171d6b210556
I20260811 19:00:32.999338 22473 ts_tablet_manager.cc:1939] T 42730b2693ef456fb43e171d6b210556 P 3bf50bf4e77b45298676246bb4db1765: Deleting consensus metadata
I20260811 19:00:33.000152 21845 catalog_manager.cc:4841] TS 3bf50bf4e77b45298676246bb4db1765 (127.13.109.1:36147): tablet 42730b2693ef456fb43e171d6b210556 (table test-table [id=522b14a9544448a3b0f71a5d05997124]) successfully deleted
I20260811 19:00:33.000350 22472 ts_tablet_manager.cc:1918] T 42730b2693ef456fb43e171d6b210556 P aa3ef1ab1dce4ff0876fed61e6719733: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:00:33.000432 22472 log.cc:1196] T 42730b2693ef456fb43e171d6b210556 P aa3ef1ab1dce4ff0876fed61e6719733: Deleting WAL directory at /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/minicluster-data/ts-2/wal/wals/42730b2693ef456fb43e171d6b210556
I20260811 19:00:33.000650 22472 ts_tablet_manager.cc:1939] T 42730b2693ef456fb43e171d6b210556 P aa3ef1ab1dce4ff0876fed61e6719733: Deleting consensus metadata
I20260811 19:00:33.000748 22476 ts_tablet_manager.cc:1918] T 42730b2693ef456fb43e171d6b210556 P 01dababe80fc41b48b6db09ba9cf2daa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:00:33.000820 22476 log.cc:1196] T 42730b2693ef456fb43e171d6b210556 P 01dababe80fc41b48b6db09ba9cf2daa: Deleting WAL directory at /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/minicluster-data/ts-1/wal/wals/42730b2693ef456fb43e171d6b210556
I20260811 19:00:33.001080 22476 ts_tablet_manager.cc:1939] T 42730b2693ef456fb43e171d6b210556 P 01dababe80fc41b48b6db09ba9cf2daa: Deleting consensus metadata
I20260811 19:00:33.001304 21845 catalog_manager.cc:4841] TS aa3ef1ab1dce4ff0876fed61e6719733 (127.13.109.3:44129): tablet 42730b2693ef456fb43e171d6b210556 (table test-table [id=522b14a9544448a3b0f71a5d05997124]) successfully deleted
I20260811 19:00:33.001734 21846 catalog_manager.cc:4841] TS 01dababe80fc41b48b6db09ba9cf2daa (127.13.109.2:34689): tablet 42730b2693ef456fb43e171d6b210556 (table test-table [id=522b14a9544448a3b0f71a5d05997124]) successfully deleted
I20260811 19:00:33.005331 22009 tablet_service.cc:1467] Processing CreateTablet for tablet 6b1f0f4399064e2f81dded23ba77463d (DEFAULT_TABLE table=test-table [id=2567bdd8449e45c99e9db8d87c5d74c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:00:33.005465 22161 tablet_service.cc:1467] Processing CreateTablet for tablet 6b1f0f4399064e2f81dded23ba77463d (DEFAULT_TABLE table=test-table [id=2567bdd8449e45c99e9db8d87c5d74c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:00:33.005618 22009 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b1f0f4399064e2f81dded23ba77463d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:00:33.005581 22316 tablet_service.cc:1467] Processing CreateTablet for tablet 6b1f0f4399064e2f81dded23ba77463d (DEFAULT_TABLE table=test-table [id=2567bdd8449e45c99e9db8d87c5d74c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:00:33.005754 22161 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b1f0f4399064e2f81dded23ba77463d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:00:33.005885 22316 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b1f0f4399064e2f81dded23ba77463d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:00:33.008281 22417 tablet_bootstrap.cc:492] T 6b1f0f4399064e2f81dded23ba77463d P 01dababe80fc41b48b6db09ba9cf2daa: Bootstrap starting.
I20260811 19:00:33.008351 22416 tablet_bootstrap.cc:492] T 6b1f0f4399064e2f81dded23ba77463d P 3bf50bf4e77b45298676246bb4db1765: Bootstrap starting.
I20260811 19:00:33.009127 22417 tablet_bootstrap.cc:654] T 6b1f0f4399064e2f81dded23ba77463d P 01dababe80fc41b48b6db09ba9cf2daa: Neither blocks nor log segments found. Creating new log.
I20260811 19:00:33.009353 22416 tablet_bootstrap.cc:654] T 6b1f0f4399064e2f81dded23ba77463d P 3bf50bf4e77b45298676246bb4db1765: Neither blocks nor log segments found. Creating new log.
I20260811 19:00:33.010038 22418 tablet_bootstrap.cc:492] T 6b1f0f4399064e2f81dded23ba77463d P aa3ef1ab1dce4ff0876fed61e6719733: Bootstrap starting.
I20260811 19:00:33.010393 22417 tablet_bootstrap.cc:492] T 6b1f0f4399064e2f81dded23ba77463d P 01dababe80fc41b48b6db09ba9cf2daa: No bootstrap required, opened a new log
I20260811 19:00:33.010493 22417 ts_tablet_manager.cc:1397] T 6b1f0f4399064e2f81dded23ba77463d P 01dababe80fc41b48b6db09ba9cf2daa: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 19:00:33.010880 22417 raft_consensus.cc:348] T 6b1f0f4399064e2f81dded23ba77463d P 01dababe80fc41b48b6db09ba9cf2daa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bf50bf4e77b45298676246bb4db1765" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 36147 } } peers { permanent_uuid: "01dababe80fc41b48b6db09ba9cf2daa" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34689 } } peers { permanent_uuid: "aa3ef1ab1dce4ff0876fed61e6719733" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 44129 } }
I20260811 19:00:33.011005 22417 raft_consensus.cc:374] T 6b1f0f4399064e2f81dded23ba77463d P 01dababe80fc41b48b6db09ba9cf2daa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:00:33.011029 22418 tablet_bootstrap.cc:654] T 6b1f0f4399064e2f81dded23ba77463d P aa3ef1ab1dce4ff0876fed61e6719733: Neither blocks nor log segments found. Creating new log.
I20260811 19:00:33.011044 22417 raft_consensus.cc:729] T 6b1f0f4399064e2f81dded23ba77463d P 01dababe80fc41b48b6db09ba9cf2daa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01dababe80fc41b48b6db09ba9cf2daa, State: Initialized, Role: FOLLOWER
I20260811 19:00:33.011166 22417 consensus_queue.cc:260] T 6b1f0f4399064e2f81dded23ba77463d P 01dababe80fc41b48b6db09ba9cf2daa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bf50bf4e77b45298676246bb4db1765" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 36147 } } peers { permanent_uuid: "01dababe80fc41b48b6db09ba9cf2daa" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34689 } } peers { permanent_uuid: "aa3ef1ab1dce4ff0876fed61e6719733" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 44129 } }
I20260811 19:00:33.011415 22417 ts_tablet_manager.cc:1428] T 6b1f0f4399064e2f81dded23ba77463d P 01dababe80fc41b48b6db09ba9cf2daa: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 19:00:33.012874 22416 tablet_bootstrap.cc:492] T 6b1f0f4399064e2f81dded23ba77463d P 3bf50bf4e77b45298676246bb4db1765: No bootstrap required, opened a new log
I20260811 19:00:33.012977 22416 ts_tablet_manager.cc:1397] T 6b1f0f4399064e2f81dded23ba77463d P 3bf50bf4e77b45298676246bb4db1765: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260811 19:00:33.013473 22416 raft_consensus.cc:348] T 6b1f0f4399064e2f81dded23ba77463d P 3bf50bf4e77b45298676246bb4db1765 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bf50bf4e77b45298676246bb4db1765" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 36147 } } peers { permanent_uuid: "01dababe80fc41b48b6db09ba9cf2daa" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34689 } } peers { permanent_uuid: "aa3ef1ab1dce4ff0876fed61e6719733" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 44129 } }
I20260811 19:00:33.013613 22416 raft_consensus.cc:374] T 6b1f0f4399064e2f81dded23ba77463d P 3bf50bf4e77b45298676246bb4db1765 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:00:33.013662 22416 raft_consensus.cc:729] T 6b1f0f4399064e2f81dded23ba77463d P 3bf50bf4e77b45298676246bb4db1765 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bf50bf4e77b45298676246bb4db1765, State: Initialized, Role: FOLLOWER
I20260811 19:00:33.013836 22416 consensus_queue.cc:260] T 6b1f0f4399064e2f81dded23ba77463d P 3bf50bf4e77b45298676246bb4db1765 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bf50bf4e77b45298676246bb4db1765" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 36147 } } peers { permanent_uuid: "01dababe80fc41b48b6db09ba9cf2daa" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34689 } } peers { permanent_uuid: "aa3ef1ab1dce4ff0876fed61e6719733" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 44129 } }
I20260811 19:00:33.014231 22416 ts_tablet_manager.cc:1428] T 6b1f0f4399064e2f81dded23ba77463d P 3bf50bf4e77b45298676246bb4db1765: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 19:00:33.014843 22418 tablet_bootstrap.cc:492] T 6b1f0f4399064e2f81dded23ba77463d P aa3ef1ab1dce4ff0876fed61e6719733: No bootstrap required, opened a new log
I20260811 19:00:33.014938 22418 ts_tablet_manager.cc:1397] T 6b1f0f4399064e2f81dded23ba77463d P aa3ef1ab1dce4ff0876fed61e6719733: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.001s
I20260811 19:00:33.015419 22418 raft_consensus.cc:348] T 6b1f0f4399064e2f81dded23ba77463d P aa3ef1ab1dce4ff0876fed61e6719733 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bf50bf4e77b45298676246bb4db1765" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 36147 } } peers { permanent_uuid: "01dababe80fc41b48b6db09ba9cf2daa" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34689 } } peers { permanent_uuid: "aa3ef1ab1dce4ff0876fed61e6719733" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 44129 } }
I20260811 19:00:33.015561 22418 raft_consensus.cc:374] T 6b1f0f4399064e2f81dded23ba77463d P aa3ef1ab1dce4ff0876fed61e6719733 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:00:33.015610 22418 raft_consensus.cc:729] T 6b1f0f4399064e2f81dded23ba77463d P aa3ef1ab1dce4ff0876fed61e6719733 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa3ef1ab1dce4ff0876fed61e6719733, State: Initialized, Role: FOLLOWER
I20260811 19:00:33.016019 22418 consensus_queue.cc:260] T 6b1f0f4399064e2f81dded23ba77463d P aa3ef1ab1dce4ff0876fed61e6719733 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bf50bf4e77b45298676246bb4db1765" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 36147 } } peers { permanent_uuid: "01dababe80fc41b48b6db09ba9cf2daa" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34689 } } peers { permanent_uuid: "aa3ef1ab1dce4ff0876fed61e6719733" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 44129 } }
I20260811 19:00:33.016498 22418 ts_tablet_manager.cc:1428] T 6b1f0f4399064e2f81dded23ba77463d P aa3ef1ab1dce4ff0876fed61e6719733: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260811 19:00:33.222683 22427 raft_consensus.cc:482] T 6b1f0f4399064e2f81dded23ba77463d P 01dababe80fc41b48b6db09ba9cf2daa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:00:33.222875 22427 raft_consensus.cc:504] T 6b1f0f4399064e2f81dded23ba77463d P 01dababe80fc41b48b6db09ba9cf2daa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bf50bf4e77b45298676246bb4db1765" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 36147 } } peers { permanent_uuid: "01dababe80fc41b48b6db09ba9cf2daa" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34689 } } peers { permanent_uuid: "aa3ef1ab1dce4ff0876fed61e6719733" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 44129 } }
I20260811 19:00:33.223366 22427 leader_election.cc:288] T 6b1f0f4399064e2f81dded23ba77463d P 01dababe80fc41b48b6db09ba9cf2daa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3bf50bf4e77b45298676246bb4db1765 (127.13.109.1:36147), aa3ef1ab1dce4ff0876fed61e6719733 (127.13.109.3:44129)
I20260811 19:00:33.223837 22030 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b1f0f4399064e2f81dded23ba77463d" candidate_uuid: "01dababe80fc41b48b6db09ba9cf2daa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bf50bf4e77b45298676246bb4db1765" is_pre_election: true
I20260811 19:00:33.223850 22337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b1f0f4399064e2f81dded23ba77463d" candidate_uuid: "01dababe80fc41b48b6db09ba9cf2daa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa3ef1ab1dce4ff0876fed61e6719733" is_pre_election: true
I20260811 19:00:33.224012 22030 raft_consensus.cc:2454] T 6b1f0f4399064e2f81dded23ba77463d P 3bf50bf4e77b45298676246bb4db1765 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01dababe80fc41b48b6db09ba9cf2daa in term 0.
I20260811 19:00:33.224048 22337 raft_consensus.cc:2454] T 6b1f0f4399064e2f81dded23ba77463d P aa3ef1ab1dce4ff0876fed61e6719733 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01dababe80fc41b48b6db09ba9cf2daa in term 0.
I20260811 19:00:33.224344 22108 leader_election.cc:302] T 6b1f0f4399064e2f81dded23ba77463d P 01dababe80fc41b48b6db09ba9cf2daa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01dababe80fc41b48b6db09ba9cf2daa, 3bf50bf4e77b45298676246bb4db1765; no voters:
I20260811 19:00:33.224548 22427 raft_consensus.cc:2781] T 6b1f0f4399064e2f81dded23ba77463d P 01dababe80fc41b48b6db09ba9cf2daa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 19:00:33.224629 22427 raft_consensus.cc:482] T 6b1f0f4399064e2f81dded23ba77463d P 01dababe80fc41b48b6db09ba9cf2daa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 19:00:33.224704 22427 raft_consensus.cc:3037] T 6b1f0f4399064e2f81dded23ba77463d P 01dababe80fc41b48b6db09ba9cf2daa [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:00:33.225958 22427 raft_consensus.cc:504] T 6b1f0f4399064e2f81dded23ba77463d P 01dababe80fc41b48b6db09ba9cf2daa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bf50bf4e77b45298676246bb4db1765" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 36147 } } peers { permanent_uuid: "01dababe80fc41b48b6db09ba9cf2daa" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34689 } } peers { permanent_uuid: "aa3ef1ab1dce4ff0876fed61e6719733" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 44129 } }
I20260811 19:00:33.226327 22427 leader_election.cc:288] T 6b1f0f4399064e2f81dded23ba77463d P 01dababe80fc41b48b6db09ba9cf2daa [CANDIDATE]: Term 1 election: Requested vote from peers 3bf50bf4e77b45298676246bb4db1765 (127.13.109.1:36147), aa3ef1ab1dce4ff0876fed61e6719733 (127.13.109.3:44129)
I20260811 19:00:33.226768 22030 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b1f0f4399064e2f81dded23ba77463d" candidate_uuid: "01dababe80fc41b48b6db09ba9cf2daa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bf50bf4e77b45298676246bb4db1765"
I20260811 19:00:33.226799 22337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b1f0f4399064e2f81dded23ba77463d" candidate_uuid: "01dababe80fc41b48b6db09ba9cf2daa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa3ef1ab1dce4ff0876fed61e6719733"
I20260811 19:00:33.226928 22337 raft_consensus.cc:3037] T 6b1f0f4399064e2f81dded23ba77463d P aa3ef1ab1dce4ff0876fed61e6719733 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:00:33.226926 22030 raft_consensus.cc:3037] T 6b1f0f4399064e2f81dded23ba77463d P 3bf50bf4e77b45298676246bb4db1765 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:00:33.227869 22337 raft_consensus.cc:2454] T 6b1f0f4399064e2f81dded23ba77463d P aa3ef1ab1dce4ff0876fed61e6719733 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01dababe80fc41b48b6db09ba9cf2daa in term 1.
I20260811 19:00:33.227949 22030 raft_consensus.cc:2454] T 6b1f0f4399064e2f81dded23ba77463d P 3bf50bf4e77b45298676246bb4db1765 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01dababe80fc41b48b6db09ba9cf2daa in term 1.
I20260811 19:00:33.228214 22108 leader_election.cc:302] T 6b1f0f4399064e2f81dded23ba77463d P 01dababe80fc41b48b6db09ba9cf2daa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01dababe80fc41b48b6db09ba9cf2daa, aa3ef1ab1dce4ff0876fed61e6719733; no voters:
I20260811 19:00:33.228435 22427 raft_consensus.cc:2781] T 6b1f0f4399064e2f81dded23ba77463d P 01dababe80fc41b48b6db09ba9cf2daa [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:00:33.228562 22427 raft_consensus.cc:686] T 6b1f0f4399064e2f81dded23ba77463d P 01dababe80fc41b48b6db09ba9cf2daa [term 1 LEADER]: Becoming Leader. State: Replica: 01dababe80fc41b48b6db09ba9cf2daa, State: Running, Role: LEADER
I20260811 19:00:33.228713 22427 consensus_queue.cc:237] T 6b1f0f4399064e2f81dded23ba77463d P 01dababe80fc41b48b6db09ba9cf2daa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bf50bf4e77b45298676246bb4db1765" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 36147 } } peers { permanent_uuid: "01dababe80fc41b48b6db09ba9cf2daa" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34689 } } peers { permanent_uuid: "aa3ef1ab1dce4ff0876fed61e6719733" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 44129 } }
I20260811 19:00:33.230463 21859 catalog_manager.cc:5495] T 6b1f0f4399064e2f81dded23ba77463d P 01dababe80fc41b48b6db09ba9cf2daa reported cstate change: term changed from 0 to 1, leader changed from <none> to 01dababe80fc41b48b6db09ba9cf2daa (127.13.109.2). New cstate: current_term: 1 leader_uuid: "01dababe80fc41b48b6db09ba9cf2daa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bf50bf4e77b45298676246bb4db1765" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 36147 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "01dababe80fc41b48b6db09ba9cf2daa" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34689 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aa3ef1ab1dce4ff0876fed61e6719733" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 44129 } health_report { overall_health: UNKNOWN } } }
I20260811 19:00:33.264434 22414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:33.260329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55194 (local address 127.13.109.2:34689)
0811 19:00:33.260475 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:33.260489 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:33.260506 (+ 17us) server_negotiation.cc:407] Connection header received
0811 19:00:33.260688 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:33.260691 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:33.260764 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:33.260846 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:00:33.261227 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:33.262056 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:33.263108 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:33.263333 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:33.263434 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:33.263757 (+ 323us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:00:33.263839 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:00:33.263905 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:33.264135 (+ 230us) server_negotiation.cc:299] Negotiation successful
0811 19:00:33.264246 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 19:00:33.272779 22030 raft_consensus.cc:1261] T 6b1f0f4399064e2f81dded23ba77463d P 3bf50bf4e77b45298676246bb4db1765 [term 1 FOLLOWER]: Refusing update from remote peer 01dababe80fc41b48b6db09ba9cf2daa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:00:33.272779 22337 raft_consensus.cc:1261] T 6b1f0f4399064e2f81dded23ba77463d P aa3ef1ab1dce4ff0876fed61e6719733 [term 1 FOLLOWER]: Refusing update from remote peer 01dababe80fc41b48b6db09ba9cf2daa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:00:33.273344 22506 consensus_queue.cc:1035] T 6b1f0f4399064e2f81dded23ba77463d P 01dababe80fc41b48b6db09ba9cf2daa [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bf50bf4e77b45298676246bb4db1765" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 36147 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:00:33.273569 22509 consensus_queue.cc:1035] T 6b1f0f4399064e2f81dded23ba77463d P 01dababe80fc41b48b6db09ba9cf2daa [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa3ef1ab1dce4ff0876fed61e6719733" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 44129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:00:33.313033 21859 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60754:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:00:33.313230 21859 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60754:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:00:33.316782 21859 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 517db3110b44426982fea15cbe648df8: Sending DeleteTablet for 3 replicas of tablet 6b1f0f4399064e2f81dded23ba77463d
I20260811 19:00:33.317668 22009 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b1f0f4399064e2f81dded23ba77463d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:00:33 UTC) from {username='oryx', principal='oryx/127.13.109.62@KRBTEST.COM'} at 127.0.0.1:39064
I20260811 19:00:33.317750 22316 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b1f0f4399064e2f81dded23ba77463d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:00:33 UTC) from {username='oryx', principal='oryx/127.13.109.62@KRBTEST.COM'} at 127.0.0.1:59116
I20260811 19:00:33.317986 22472 tablet_replica.cc:330] T 6b1f0f4399064e2f81dded23ba77463d P aa3ef1ab1dce4ff0876fed61e6719733: stopping tablet replica
I20260811 19:00:33.317987 22473 tablet_replica.cc:330] T 6b1f0f4399064e2f81dded23ba77463d P 3bf50bf4e77b45298676246bb4db1765: stopping tablet replica
I20260811 19:00:33.318137 22473 raft_consensus.cc:2229] T 6b1f0f4399064e2f81dded23ba77463d P 3bf50bf4e77b45298676246bb4db1765 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:00:33.318137 22472 raft_consensus.cc:2229] T 6b1f0f4399064e2f81dded23ba77463d P aa3ef1ab1dce4ff0876fed61e6719733 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:00:33.318292 22472 raft_consensus.cc:2258] T 6b1f0f4399064e2f81dded23ba77463d P aa3ef1ab1dce4ff0876fed61e6719733 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:00:33.318288 22473 raft_consensus.cc:2258] T 6b1f0f4399064e2f81dded23ba77463d P 3bf50bf4e77b45298676246bb4db1765 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:00:33.318272 22159 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b1f0f4399064e2f81dded23ba77463d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:00:33 UTC) from {username='oryx', principal='oryx/127.13.109.62@KRBTEST.COM'} at 127.0.0.1:55168
I20260811 19:00:33.318606 22476 tablet_replica.cc:330] T 6b1f0f4399064e2f81dded23ba77463d P 01dababe80fc41b48b6db09ba9cf2daa: stopping tablet replica
I20260811 19:00:33.318799 22476 raft_consensus.cc:2229] T 6b1f0f4399064e2f81dded23ba77463d P 01dababe80fc41b48b6db09ba9cf2daa [term 1 LEADER]: Raft consensus shutting down.
I20260811 19:00:33.319048 22476 raft_consensus.cc:2258] T 6b1f0f4399064e2f81dded23ba77463d P 01dababe80fc41b48b6db09ba9cf2daa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:00:33.319141 22473 ts_tablet_manager.cc:1905] T 6b1f0f4399064e2f81dded23ba77463d P 3bf50bf4e77b45298676246bb4db1765: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:00:33.319231 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 21940
I20260811 19:00:33.324575 22476 ts_tablet_manager.cc:1905] T 6b1f0f4399064e2f81dded23ba77463d P 01dababe80fc41b48b6db09ba9cf2daa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:00:33.325387 22472 ts_tablet_manager.cc:1905] T 6b1f0f4399064e2f81dded23ba77463d P aa3ef1ab1dce4ff0876fed61e6719733: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:00:33.328104 22472 ts_tablet_manager.cc:1918] T 6b1f0f4399064e2f81dded23ba77463d P aa3ef1ab1dce4ff0876fed61e6719733: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 19:00:33.328241 22472 log.cc:1196] T 6b1f0f4399064e2f81dded23ba77463d P aa3ef1ab1dce4ff0876fed61e6719733: Deleting WAL directory at /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/minicluster-data/ts-2/wal/wals/6b1f0f4399064e2f81dded23ba77463d
I20260811 19:00:33.328532 22472 ts_tablet_manager.cc:1939] T 6b1f0f4399064e2f81dded23ba77463d P aa3ef1ab1dce4ff0876fed61e6719733: Deleting consensus metadata
I20260811 19:00:33.328903 22476 ts_tablet_manager.cc:1918] T 6b1f0f4399064e2f81dded23ba77463d P 01dababe80fc41b48b6db09ba9cf2daa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 19:00:33.329030 22476 log.cc:1196] T 6b1f0f4399064e2f81dded23ba77463d P 01dababe80fc41b48b6db09ba9cf2daa: Deleting WAL directory at /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474792496492-13748-0/minicluster-data/ts-1/wal/wals/6b1f0f4399064e2f81dded23ba77463d
I20260811 19:00:33.329373 22476 ts_tablet_manager.cc:1939] T 6b1f0f4399064e2f81dded23ba77463d P 01dababe80fc41b48b6db09ba9cf2daa: Deleting consensus metadata
I20260811 19:00:33.329759 21845 catalog_manager.cc:4841] TS aa3ef1ab1dce4ff0876fed61e6719733 (127.13.109.3:44129): tablet 6b1f0f4399064e2f81dded23ba77463d (table test-table [id=2567bdd8449e45c99e9db8d87c5d74c4]) successfully deleted
I20260811 19:00:33.330708 21846 catalog_manager.cc:4841] TS 01dababe80fc41b48b6db09ba9cf2daa (127.13.109.2:34689): tablet 6b1f0f4399064e2f81dded23ba77463d (table test-table [id=2567bdd8449e45c99e9db8d87c5d74c4]) successfully deleted
W20260811 19:00:33.332978 21845 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.109.1:36147 (error 108)
I20260811 19:00:33.333935 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 22090
I20260811 19:00:33.338603 22515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:33.333936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.1:36147 (local address 127.0.0.1:39070)
0811 19:00:33.338287 (+ 4351us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:33.338428 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.1:36147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4217,"thread_start_us":85,"threads_started":1}
W20260811 19:00:33.338869 21845 catalog_manager.cc:4584] TS 3bf50bf4e77b45298676246bb4db1765 (127.13.109.1:36147): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6b1f0f4399064e2f81dded23ba77463d: Network error: Client connection negotiation failed: client connection to 127.13.109.1:36147: connect: Connection refused (error 111)
I20260811 19:00:33.349192 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 22242
I20260811 19:00:33.360139 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 21827
2026-08-11T19:00:33Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3685 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786474792496492-13748-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:00:33 dist-test-slave-0kls krb5kdc[22518](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:00:33 dist-test-slave-0kls krb5kdc[22518](info): set up 2 sockets
Aug 11 19:00:33 dist-test-slave-0kls krb5kdc[22518](info): commencing operation
krb5kdc: starting...
W20260811 19:00:34.483115 13748 mini_kdc.cc:121] Time spent starting KDC: real 1.096s user 0.004s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786474792496492-13748-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786474792496492-13748-0/krb5kdc/test-user.keytab.
Aug 11 19:00:34 dist-test-slave-0kls krb5kdc[22518](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474834, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:00:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:00:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.109.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.109.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.109.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab.
Entry for principal kudu/127.13.109.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab.
I20260811 19:00:34.620718 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786474792496492-13748-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786474792496492-13748-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786474792496492-13748-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.62:35233
--webserver_interface=127.13.109.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab
--principal=kudu/127.13.109.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:36095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.109.62:35233
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:00:34.744822 22533 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:00:34.745182 22533 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:00:34.745270 22533 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:00:34.749969 22533 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:00:34.750082 22533 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:00:34.750124 22533 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:00:34.750187 22533 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:00:34.750226 22533 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:00:34.755856 22533 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:36095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786474792496492-13748-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786474792496492-13748-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.109.62:35233
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.62:35233
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.109.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22533
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786474792496492-13748-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:00:34.757038 22533 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:00:34.758184 22533 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:00:34.767364 22539 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:34.767400 22542 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:34.767392 22540 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:34.767745 22533 server_base.cc:1034] running on GCE node
I20260811 19:00:34.768538 22533 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:00:34.769873 22533 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:00:34.771059 22533 hybrid_clock.cc:648] HybridClock initialized: now 1786474834771050 us; error 34 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-11T19:00:34Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1426 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:00:34 dist-test-slave-0kls krb5kdc[22547](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:00:34 dist-test-slave-0kls krb5kdc[22547](info): set up 2 sockets
Aug 11 19:00:34 dist-test-slave-0kls krb5kdc[22547](info): commencing operation
krb5kdc: starting...
W20260811 19:00:35.902148 13748 mini_kdc.cc:121] Time spent starting KDC: real 1.088s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/krb5kdc/test-user.keytab.
Aug 11 19:00:35 dist-test-slave-0kls krb5kdc[22547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474835, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:00:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:00:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.109.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.109.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.109.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab.
Entry for principal kudu/127.13.109.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab.
I20260811 19:00:36.054396 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.62:43557
--webserver_interface=127.13.109.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab
--principal=kudu/127.13.109.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:36873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.109.62:43557
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:00:36.178740 22560 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:00:36.179064 22560 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:00:36.179126 22560 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:00:36.183387 22560 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:00:36.183463 22560 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:00:36.183485 22560 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:00:36.183502 22560 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:00:36.183518 22560 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:00:36.189270 22560 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:36873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.109.62:43557
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab
--principal=kudu/127.13.109.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.62:43557
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.109.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22560
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:00:36.190532 22560 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:00:36.191630 22560 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:00:36.201220 22569 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:36.201313 22560 server_base.cc:1034] running on GCE node
W20260811 19:00:36.201251 22566 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:36.201254 22567 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:36.202219 22560 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:00:36.203469 22560 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:00:36.204680 22560 hybrid_clock.cc:648] HybridClock initialized: now 1786474836204660 us; error 35 us; skew 500 ppm
Aug 11 19:00:36 dist-test-slave-0kls krb5kdc[22547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474836, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:00:36.207747 22560 init.cc:377] Logged in from keytab as kudu/127.13.109.62@KRBTEST.COM (short username kudu)
I20260811 19:00:36.208842 22560 webserver.cc:460] Webserver started at http://127.13.109.62:40759/ using document root <none> and password file <none>
I20260811 19:00:36.209506 22560 fs_manager.cc:359] Metadata directory not provided
I20260811 19:00:36.209613 22560 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:00:36.209956 22560 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:00:36.212387 22560 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/master-0/data/instance:
uuid: "9086e30907ac40d189c9847ea24713a2"
format_stamp: "Formatted at 2026-08-11 19:00:36 on dist-test-slave-0kls"
server_key: "45914f2ce1590d12e4d4a230fe45c385"
server_key_iv: "534b2ef3186e8d3a3c0a803ce88dfff9"
server_key_version: "encryptionkey@0"
I20260811 19:00:36.213147 22560 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/master-0/wal/instance:
uuid: "9086e30907ac40d189c9847ea24713a2"
format_stamp: "Formatted at 2026-08-11 19:00:36 on dist-test-slave-0kls"
server_key: "45914f2ce1590d12e4d4a230fe45c385"
server_key_iv: "534b2ef3186e8d3a3c0a803ce88dfff9"
server_key_version: "encryptionkey@0"
I20260811 19:00:36.217732 22560 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260811 19:00:36.221303 22576 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:36.222921 22560 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 19:00:36.223120 22560 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/master-0/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/master-0/wal
uuid: "9086e30907ac40d189c9847ea24713a2"
format_stamp: "Formatted at 2026-08-11 19:00:36 on dist-test-slave-0kls"
server_key: "45914f2ce1590d12e4d4a230fe45c385"
server_key_iv: "534b2ef3186e8d3a3c0a803ce88dfff9"
server_key_version: "encryptionkey@0"
I20260811 19:00:36.223271 22560 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:00:36.252763 22560 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:00:36.256825 22560 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:00:36.257113 22560 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:00:36.266474 22560 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.62:43557
I20260811 19:00:36.266546 22634 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.62:43557 every 8 connection(s)
I20260811 19:00:36.267761 22560 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
I20260811 19:00:36.271729 22635 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:00:36.274324 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 22560
I20260811 19:00:36.274526 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/master-0/wal/instance
I20260811 19:00:36.274928 13748 external_mini_cluster.cc:1428] Setting key 6fbb6506cb732738cefe881ad46fe9af
I20260811 19:00:36.280205 22635 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9086e30907ac40d189c9847ea24713a2: Bootstrap starting.
I20260811 19:00:36.284562 22635 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9086e30907ac40d189c9847ea24713a2: Neither blocks nor log segments found. Creating new log.
Aug 11 19:00:36 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474835, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
I20260811 19:00:36.286223 22635 log.cc:824] T 00000000000000000000000000000000 P 9086e30907ac40d189c9847ea24713a2: Log is configured to *not* fsync() on all Append() calls
I20260811 19:00:36.289541 22635 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9086e30907ac40d189c9847ea24713a2: No bootstrap required, opened a new log
I20260811 19:00:36.293711 22638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:36.276807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40210 (local address 127.13.109.62:43557)
0811 19:00:36.277387 (+ 580us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:36.277414 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:36.277465 (+ 51us) server_negotiation.cc:407] Connection header received
0811 19:00:36.278333 (+ 868us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:36.278357 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:36.278784 (+ 427us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:36.279213 (+ 429us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:36.280167 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:36.281299 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:36.282362 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:36.282922 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:36.283142 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:36.287002 (+ 3860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:36.287010 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:36.287035 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:36.287070 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:36.289352 (+ 2282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:36.290108 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:36.290117 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:36.290128 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:36.290314 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:36.290719 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:36.290723 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:36.290725 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:36.291368 (+ 643us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:36.291590 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:36.291961 (+ 371us) server_negotiation.cc:299] Negotiation successful
0811 19:00:36.292370 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":400,"thread_start_us":185,"threads_started":1}
I20260811 19:00:36.295125 22635 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9086e30907ac40d189c9847ea24713a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9086e30907ac40d189c9847ea24713a2" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 43557 } }
I20260811 19:00:36.295475 22635 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9086e30907ac40d189c9847ea24713a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:00:36.295547 22635 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9086e30907ac40d189c9847ea24713a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9086e30907ac40d189c9847ea24713a2, State: Initialized, Role: FOLLOWER
I20260811 19:00:36.296083 22635 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9086e30907ac40d189c9847ea24713a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9086e30907ac40d189c9847ea24713a2" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 43557 } }
I20260811 19:00:36.296307 22635 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9086e30907ac40d189c9847ea24713a2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:00:36.296417 22635 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9086e30907ac40d189c9847ea24713a2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:00:36.296545 22635 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9086e30907ac40d189c9847ea24713a2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:00:36.297914 22635 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9086e30907ac40d189c9847ea24713a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9086e30907ac40d189c9847ea24713a2" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 43557 } }
I20260811 19:00:36.298367 22635 leader_election.cc:302] T 00000000000000000000000000000000 P 9086e30907ac40d189c9847ea24713a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9086e30907ac40d189c9847ea24713a2; no voters:
I20260811 19:00:36.298838 22635 leader_election.cc:288] T 00000000000000000000000000000000 P 9086e30907ac40d189c9847ea24713a2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:00:36.299106 22640 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9086e30907ac40d189c9847ea24713a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:00:36.299513 22640 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9086e30907ac40d189c9847ea24713a2 [term 1 LEADER]: Becoming Leader. State: Replica: 9086e30907ac40d189c9847ea24713a2, State: Running, Role: LEADER
I20260811 19:00:36.300243 22640 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9086e30907ac40d189c9847ea24713a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9086e30907ac40d189c9847ea24713a2" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 43557 } }
I20260811 19:00:36.300653 22635 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9086e30907ac40d189c9847ea24713a2 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:00:36.302464 22641 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9086e30907ac40d189c9847ea24713a2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9086e30907ac40d189c9847ea24713a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9086e30907ac40d189c9847ea24713a2" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 43557 } } }
I20260811 19:00:36.302517 22642 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9086e30907ac40d189c9847ea24713a2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9086e30907ac40d189c9847ea24713a2. Latest consensus state: current_term: 1 leader_uuid: "9086e30907ac40d189c9847ea24713a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9086e30907ac40d189c9847ea24713a2" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 43557 } } }
I20260811 19:00:36.302624 22641 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9086e30907ac40d189c9847ea24713a2 [sys.catalog]: This master's current role is: LEADER
I20260811 19:00:36.302662 22642 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9086e30907ac40d189c9847ea24713a2 [sys.catalog]: This master's current role is: LEADER
I20260811 19:00:36.303565 22653 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:00:36.307701 22653 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:00:36.315094 22653 catalog_manager.cc:1347] Generated new cluster ID: 29443ed4862f4e0f93c78996e43f60eb
I20260811 19:00:36.315234 22653 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:00:36.329150 22653 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:00:36.330752 22653 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:00:36.341830 22653 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9086e30907ac40d189c9847ea24713a2: Generated new TSK 0
I20260811 19:00:36.342805 22653 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.109.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.109.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.109.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.1.keytab.
Entry for principal kudu/127.13.109.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.1.keytab.
I20260811 19:00:36.385375 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.1:0
--local_ip_for_outbound_sockets=127.13.109.1
--webserver_interface=127.13.109.1
--webserver_port=0
--tserver_master_addrs=127.13.109.62:43557
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.1.keytab
--principal=kudu/127.13.109.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:36873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:00:36.530469 22661 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:00:36.530880 22661 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:00:36.530963 22661 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:00:36.535710 22661 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:00:36.535848 22661 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:00:36.535967 22661 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.1
I20260811 19:00:36.542608 22661 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:36873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.1.keytab
--principal=kudu/127.13.109.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.109.1
--webserver_port=0
--tserver_master_addrs=127.13.109.62:43557
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22661
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.1
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:00:36.544272 22661 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:00:36.545584 22661 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 19:00:36.558000 22661 server_base.cc:1034] running on GCE node
W20260811 19:00:36.557921 22672 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:36.557924 22668 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:36.557921 22667 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:36.558830 22661 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:00:36.559703 22661 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:00:36.560964 22661 hybrid_clock.cc:648] HybridClock initialized: now 1786474836560919 us; error 62 us; skew 500 ppm
Aug 11 19:00:36 dist-test-slave-0kls krb5kdc[22547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474836, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:00:36.564853 22661 init.cc:377] Logged in from keytab as kudu/127.13.109.1@KRBTEST.COM (short username kudu)
I20260811 19:00:36.566275 22661 webserver.cc:460] Webserver started at http://127.13.109.1:44183/ using document root <none> and password file <none>
I20260811 19:00:36.567081 22661 fs_manager.cc:359] Metadata directory not provided
I20260811 19:00:36.567165 22661 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:00:36.567444 22661 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:00:36.570271 22661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/ts-0/data/instance:
uuid: "b325635d075b4d769880d77492ccf776"
format_stamp: "Formatted at 2026-08-11 19:00:36 on dist-test-slave-0kls"
server_key: "db4ff824b6059db2eb5b06580d1527b6"
server_key_iv: "cf3f3205ce659e880787d6bec0103bde"
server_key_version: "encryptionkey@0"
I20260811 19:00:36.571156 22661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/ts-0/wal/instance:
uuid: "b325635d075b4d769880d77492ccf776"
format_stamp: "Formatted at 2026-08-11 19:00:36 on dist-test-slave-0kls"
server_key: "db4ff824b6059db2eb5b06580d1527b6"
server_key_iv: "cf3f3205ce659e880787d6bec0103bde"
server_key_version: "encryptionkey@0"
I20260811 19:00:36.576153 22661 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 19:00:36.579909 22682 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:36.581933 22661 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 19:00:36.582134 22661 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/ts-0/wal
uuid: "b325635d075b4d769880d77492ccf776"
format_stamp: "Formatted at 2026-08-11 19:00:36 on dist-test-slave-0kls"
server_key: "db4ff824b6059db2eb5b06580d1527b6"
server_key_iv: "cf3f3205ce659e880787d6bec0103bde"
server_key_version: "encryptionkey@0"
I20260811 19:00:36.582307 22661 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:00:36.602089 22661 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:00:36.606254 22661 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:00:36.606493 22661 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:00:36.607194 22661 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:00:36.608206 22661 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:00:36.608297 22661 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:36.608393 22661 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:00:36.608443 22661 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:36.620167 22661 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.1:42857
I20260811 19:00:36.620261 22805 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.1:42857 every 8 connection(s)
I20260811 19:00:36.621407 22661 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
I20260811 19:00:36.626144 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 22661
I20260811 19:00:36.626394 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/ts-0/wal/instance
I20260811 19:00:36.626802 13748 external_mini_cluster.cc:1428] Setting key f165d20e9c2fb798c1712c72273f0d9c
Aug 11 19:00:36 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474836, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.1@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.109.2@KRBTEST.COM; defaulting to no policy
I20260811 19:00:36.643370 22638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:36.624848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.1:32805 (local address 127.13.109.62:43557)
0811 19:00:36.625077 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:36.625082 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:36.626827 (+ 1745us) server_negotiation.cc:407] Connection header received
0811 19:00:36.629423 (+ 2596us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:36.629427 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:36.629493 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:36.629581 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:36.631306 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:36.631907 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:36.633880 (+ 1973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:36.634149 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:36.634265 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:36.638123 (+ 3858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:36.638127 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:36.638129 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:36.638181 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:36.640304 (+ 2123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:36.641162 (+ 858us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:36.641179 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:36.641180 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:36.641356 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:36.641918 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:36.641922 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:36.641924 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:36.642177 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:36.642309 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:36.642982 (+ 673us) server_negotiation.cc:299] Negotiation successful
0811 19:00:36.643155 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260811 19:00:36.644431 22808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:36.625421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:43557 (local address 127.13.109.1:32805)
0811 19:00:36.626526 (+ 1105us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:36.626637 (+ 111us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:36.629087 (+ 2450us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:36.629925 (+ 838us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:36.629937 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:36.630452 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:36.631103 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:36.631122 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:36.632067 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:36.632072 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:36.633609 (+ 1537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:36.633638 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:36.634365 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:36.634373 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:36.634647 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:36.635444 (+ 797us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:36.635473 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:36.637832 (+ 2359us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:00:36.640495 (+ 2663us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:36.640505 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:36.640532 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:36.641011 (+ 479us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:36.641489 (+ 478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:36.641492 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:36.641495 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:36.641731 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:36.642313 (+ 582us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:00:36.642324 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:00:36.642679 (+ 355us) client_negotiation.cc:769] Sending connection context
0811 19:00:36.642918 (+ 239us) client_negotiation.cc:240] Negotiation successful
0811 19:00:36.643367 (+ 449us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":293,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.109.2@KRBTEST.COM" created.
I20260811 19:00:36.646236 22806 heartbeater.cc:347] Connected to a master server at 127.13.109.62:43557
I20260811 19:00:36.646591 22806 heartbeater.cc:464] Registering TS with master...
I20260811 19:00:36.647627 22806 heartbeater.cc:511] Master 127.13.109.62:43557 requested a full tablet report, sending...
I20260811 19:00:36.649971 22595 ts_manager.cc:194] Registered new tserver with Master: b325635d075b4d769880d77492ccf776 (127.13.109.1:42857)
I20260811 19:00:36.652218 22595 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.109.1@KRBTEST.COM'} at 127.13.109.1:32805
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.109.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.2.keytab.
Entry for principal kudu/127.13.109.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.2.keytab.
I20260811 19:00:36.672323 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.2:0
--local_ip_for_outbound_sockets=127.13.109.2
--webserver_interface=127.13.109.2
--webserver_port=0
--tserver_master_addrs=127.13.109.62:43557
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.2.keytab
--principal=kudu/127.13.109.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:36873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:00:36.812085 22811 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:00:36.812482 22811 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:00:36.812564 22811 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:00:36.817147 22811 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:00:36.817276 22811 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:00:36.817377 22811 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.2
I20260811 19:00:36.823503 22811 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:36873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.2.keytab
--principal=kudu/127.13.109.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.109.2
--webserver_port=0
--tserver_master_addrs=127.13.109.62:43557
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22811
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.2
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:00:36.824822 22811 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:00:36.825995 22811 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:00:36.837477 22819 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:36.837477 22818 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:36.837683 22811 server_base.cc:1034] running on GCE node
W20260811 19:00:36.837508 22821 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:36.838318 22811 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:00:36.839108 22811 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:00:36.840318 22811 hybrid_clock.cc:648] HybridClock initialized: now 1786474836840289 us; error 39 us; skew 500 ppm
Aug 11 19:00:36 dist-test-slave-0kls krb5kdc[22547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474836, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:00:36.843294 22811 init.cc:377] Logged in from keytab as kudu/127.13.109.2@KRBTEST.COM (short username kudu)
I20260811 19:00:36.844297 22811 webserver.cc:460] Webserver started at http://127.13.109.2:44653/ using document root <none> and password file <none>
I20260811 19:00:36.844911 22811 fs_manager.cc:359] Metadata directory not provided
I20260811 19:00:36.845009 22811 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:00:36.845273 22811 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:00:36.847575 22811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/ts-1/data/instance:
uuid: "fb39a76844e4427c98abd64284967187"
format_stamp: "Formatted at 2026-08-11 19:00:36 on dist-test-slave-0kls"
server_key: "6e78b7e7034b3464c901e6d7c74c6d4e"
server_key_iv: "ee5b4d4f71f2e8834eea830347cd7ced"
server_key_version: "encryptionkey@0"
I20260811 19:00:36.848204 22811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/ts-1/wal/instance:
uuid: "fb39a76844e4427c98abd64284967187"
format_stamp: "Formatted at 2026-08-11 19:00:36 on dist-test-slave-0kls"
server_key: "6e78b7e7034b3464c901e6d7c74c6d4e"
server_key_iv: "ee5b4d4f71f2e8834eea830347cd7ced"
server_key_version: "encryptionkey@0"
I20260811 19:00:36.852409 22811 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 19:00:36.855229 22829 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:36.856639 22811 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 19:00:36.856808 22811 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/ts-1/wal
uuid: "fb39a76844e4427c98abd64284967187"
format_stamp: "Formatted at 2026-08-11 19:00:36 on dist-test-slave-0kls"
server_key: "6e78b7e7034b3464c901e6d7c74c6d4e"
server_key_iv: "ee5b4d4f71f2e8834eea830347cd7ced"
server_key_version: "encryptionkey@0"
I20260811 19:00:36.856946 22811 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:00:36.874020 22811 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:00:36.877578 22811 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:00:36.877874 22811 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:00:36.878557 22811 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:00:36.879477 22811 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:00:36.879558 22811 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:36.879638 22811 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:00:36.879694 22811 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:36.888648 22811 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.2:35713
I20260811 19:00:36.888710 22949 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.2:35713 every 8 connection(s)
I20260811 19:00:36.889745 22811 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
I20260811 19:00:36.892086 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 22811
I20260811 19:00:36.892225 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/ts-1/wal/instance
I20260811 19:00:36.892529 13748 external_mini_cluster.cc:1428] Setting key 44529dcd29611e4ee32bccfded664764
Aug 11 19:00:36 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474836, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.2@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.109.3@KRBTEST.COM; defaulting to no policy
I20260811 19:00:36.905359 22638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:36.891933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.2:58979 (local address 127.13.109.62:43557)
0811 19:00:36.892120 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:36.892126 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:36.893222 (+ 1096us) server_negotiation.cc:407] Connection header received
0811 19:00:36.894692 (+ 1470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:36.894698 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:36.894762 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:36.894859 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:36.896111 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:36.896663 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:36.898021 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:36.898278 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:36.898401 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:36.901117 (+ 2716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:36.901121 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:36.901123 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:36.901154 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:36.902905 (+ 1751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:36.903553 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:36.903557 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:36.903559 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:36.903667 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:36.904053 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:36.904056 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:36.904058 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:36.904285 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:36.904401 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:36.905031 (+ 630us) server_negotiation.cc:299] Negotiation successful
0811 19:00:36.905179 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 19:00:36.906539 22953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:36.892457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:43557 (local address 127.13.109.2:58979)
0811 19:00:36.893031 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:36.893113 (+ 82us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:36.894466 (+ 1353us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:36.895047 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:36.895054 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:36.895440 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:36.895948 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:36.895961 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:36.896806 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:36.896810 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:36.897868 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:36.897881 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:36.898455 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:36.898459 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:36.898655 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:36.899312 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:36.899330 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:36.900910 (+ 1580us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:00:36.903075 (+ 2165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:36.903092 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:36.903114 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:36.903424 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:36.903772 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:36.903775 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:36.903777 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:36.903947 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:36.904442 (+ 495us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:00:36.904455 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:00:36.904759 (+ 304us) client_negotiation.cc:769] Sending connection context
0811 19:00:36.904986 (+ 227us) client_negotiation.cc:240] Negotiation successful
0811 19:00:36.905381 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":183,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.109.3@KRBTEST.COM" created.
I20260811 19:00:36.907987 22950 heartbeater.cc:347] Connected to a master server at 127.13.109.62:43557
I20260811 19:00:36.908263 22950 heartbeater.cc:464] Registering TS with master...
I20260811 19:00:36.908838 22950 heartbeater.cc:511] Master 127.13.109.62:43557 requested a full tablet report, sending...
I20260811 19:00:36.910033 22595 ts_manager.cc:194] Registered new tserver with Master: fb39a76844e4427c98abd64284967187 (127.13.109.2:35713)
I20260811 19:00:36.910964 22595 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.109.2@KRBTEST.COM'} at 127.13.109.2:58979
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.109.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.3.keytab.
Entry for principal kudu/127.13.109.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.3.keytab.
I20260811 19:00:36.925463 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.3:0
--local_ip_for_outbound_sockets=127.13.109.3
--webserver_interface=127.13.109.3
--webserver_port=0
--tserver_master_addrs=127.13.109.62:43557
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.3.keytab
--principal=kudu/127.13.109.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:36873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:00:37.073848 22956 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:00:37.074280 22956 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:00:37.074335 22956 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:00:37.079269 22956 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:00:37.079373 22956 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:00:37.079447 22956 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.3
I20260811 19:00:37.087435 22956 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:36873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.3.keytab
--principal=kudu/127.13.109.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.109.3
--webserver_port=0
--tserver_master_addrs=127.13.109.62:43557
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22956
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.3
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:00:37.089426 22956 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:00:37.091413 22956 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:00:37.105161 22961 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:37.105151 22963 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:37.105157 22965 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:37.105536 22956 server_base.cc:1034] running on GCE node
I20260811 19:00:37.106189 22956 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:00:37.107067 22956 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:00:37.108311 22956 hybrid_clock.cc:648] HybridClock initialized: now 1786474837108271 us; error 64 us; skew 500 ppm
Aug 11 19:00:37 dist-test-slave-0kls krb5kdc[22547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474837, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:00:37.111913 22956 init.cc:377] Logged in from keytab as kudu/127.13.109.3@KRBTEST.COM (short username kudu)
I20260811 19:00:37.113077 22956 webserver.cc:460] Webserver started at http://127.13.109.3:38985/ using document root <none> and password file <none>
I20260811 19:00:37.113888 22956 fs_manager.cc:359] Metadata directory not provided
I20260811 19:00:37.114001 22956 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:00:37.114300 22956 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:00:37.116875 22956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/ts-2/data/instance:
uuid: "66a7b20a76754d5d913d2c0572d763bb"
format_stamp: "Formatted at 2026-08-11 19:00:37 on dist-test-slave-0kls"
server_key: "6c7dccb0076ef8fdd3f813aaba0bdc10"
server_key_iv: "7b37c608804df29569c60d4f9d8d35d8"
server_key_version: "encryptionkey@0"
I20260811 19:00:37.117590 22956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/ts-2/wal/instance:
uuid: "66a7b20a76754d5d913d2c0572d763bb"
format_stamp: "Formatted at 2026-08-11 19:00:37 on dist-test-slave-0kls"
server_key: "6c7dccb0076ef8fdd3f813aaba0bdc10"
server_key_iv: "7b37c608804df29569c60d4f9d8d35d8"
server_key_version: "encryptionkey@0"
I20260811 19:00:37.123142 22956 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 19:00:37.127120 22973 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:37.129066 22956 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 19:00:37.129277 22956 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/ts-2/wal
uuid: "66a7b20a76754d5d913d2c0572d763bb"
format_stamp: "Formatted at 2026-08-11 19:00:37 on dist-test-slave-0kls"
server_key: "6c7dccb0076ef8fdd3f813aaba0bdc10"
server_key_iv: "7b37c608804df29569c60d4f9d8d35d8"
server_key_version: "encryptionkey@0"
I20260811 19:00:37.129441 22956 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:00:37.159036 22956 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:00:37.165026 22956 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:00:37.165345 22956 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:00:37.166262 22956 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:00:37.168022 22956 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:00:37.168146 22956 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:37.168246 22956 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:00:37.168292 22956 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:37.179685 22956 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.3:42513
I20260811 19:00:37.179703 23092 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.3:42513 every 8 connection(s)
I20260811 19:00:37.180869 22956 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
I20260811 19:00:37.186542 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 22956
I20260811 19:00:37.186705 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/ts-2/wal/instance
I20260811 19:00:37.187019 13748 external_mini_cluster.cc:1428] Setting key 4657e69a2d44d2d7f9d239809021f63a
Aug 11 19:00:37 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474837, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.3@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
I20260811 19:00:37.196893 22638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:37.183214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.3:39253 (local address 127.13.109.62:43557)
0811 19:00:37.183387 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:37.183391 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:37.184607 (+ 1216us) server_negotiation.cc:407] Connection header received
0811 19:00:37.185602 (+ 995us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:37.185606 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:37.185671 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:37.185785 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:37.187589 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:37.188296 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:37.189978 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:37.190214 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:37.190334 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:37.193061 (+ 2727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:37.193064 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:37.193067 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:37.193097 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:37.194532 (+ 1435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:37.195153 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:37.195158 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:37.195161 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:37.195247 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:37.195631 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:37.195636 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:37.195638 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:37.195851 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:37.195967 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:37.196557 (+ 590us) server_negotiation.cc:299] Negotiation successful
0811 19:00:37.196717 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 19:00:37.197947 23095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:37.183649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:43557 (local address 127.13.109.3:39253)
0811 19:00:37.184422 (+ 773us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:37.184468 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:37.185356 (+ 888us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:37.186015 (+ 659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:37.186022 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:37.186603 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:37.187411 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:37.187431 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:37.188426 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:37.188429 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:37.189778 (+ 1349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:37.189843 (+ 65us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:37.190384 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:37.190389 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:37.190599 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:37.191207 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:37.191227 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:37.192829 (+ 1602us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:00:37.194705 (+ 1876us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:37.194712 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:37.194734 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:37.195036 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:37.195352 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:37.195355 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:37.195357 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:37.195521 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:37.195967 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:00:37.195978 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:00:37.196293 (+ 315us) client_negotiation.cc:769] Sending connection context
0811 19:00:37.196518 (+ 225us) client_negotiation.cc:240] Negotiation successful
0811 19:00:37.196906 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":562,"thread_start_us":127,"threads_started":1}
I20260811 19:00:37.199312 23093 heartbeater.cc:347] Connected to a master server at 127.13.109.62:43557
I20260811 19:00:37.199574 23093 heartbeater.cc:464] Registering TS with master...
I20260811 19:00:37.200264 23093 heartbeater.cc:511] Master 127.13.109.62:43557 requested a full tablet report, sending...
I20260811 19:00:37.201642 22594 ts_manager.cc:194] Registered new tserver with Master: 66a7b20a76754d5d913d2c0572d763bb (127.13.109.3:42513)
I20260811 19:00:37.202436 22594 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.109.3@KRBTEST.COM'} at 127.13.109.3:39253
I20260811 19:00:37.214324 13748 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.13.109.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.109.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.109.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/krb5kdc/oryx_127.13.109.1.keytab.
Entry for principal oryx/127.13.109.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/krb5kdc/oryx_127.13.109.1.keytab.
WARNING: no policy specified for oryx/127.13.109.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.109.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.109.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/krb5kdc/oryx_127.13.109.2.keytab.
Entry for principal oryx/127.13.109.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/krb5kdc/oryx_127.13.109.2.keytab.
WARNING: no policy specified for oryx/127.13.109.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.109.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.109.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/krb5kdc/oryx_127.13.109.3.keytab.
Entry for principal oryx/127.13.109.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/krb5kdc/oryx_127.13.109.3.keytab.
I20260811 19:00:37.302294 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 22661
I20260811 19:00:37.311574 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 22811
I20260811 19:00:37.321149 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 22956
I20260811 19:00:37.329777 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 22560
I20260811 19:00:37.339023 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.62:43557
--webserver_interface=127.13.109.62
--webserver_port=40759
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab
--principal=kudu/127.13.109.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:36873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.109.62:43557
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:00:37.459272 23105 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:00:37.459661 23105 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:00:37.459758 23105 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:00:37.464162 23105 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:00:37.464310 23105 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:00:37.464349 23105 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:00:37.464389 23105 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:00:37.464426 23105 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:00:37.470371 23105 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:36873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.109.62:43557
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab
--principal=kudu/127.13.109.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.62:43557
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.109.62
--webserver_port=40759
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23105
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:00:37.471748 23105 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:00:37.473009 23105 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:00:37.482748 23110 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:37.482851 23111 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:37.482833 23105 server_base.cc:1034] running on GCE node
W20260811 19:00:37.482775 23113 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:37.483681 23105 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:00:37.484974 23105 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:00:37.486151 23105 hybrid_clock.cc:648] HybridClock initialized: now 1786474837486114 us; error 49 us; skew 500 ppm
Aug 11 19:00:37 dist-test-slave-0kls krb5kdc[22547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474837, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:00:37.489192 23105 init.cc:377] Logged in from keytab as kudu/127.13.109.62@KRBTEST.COM (short username kudu)
I20260811 19:00:37.490293 23105 webserver.cc:460] Webserver started at http://127.13.109.62:40759/ using document root <none> and password file <none>
I20260811 19:00:37.490923 23105 fs_manager.cc:359] Metadata directory not provided
I20260811 19:00:37.491052 23105 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:00:37.495479 23105 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 19:00:37.498063 23122 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:37.499576 23105 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 19:00:37.499799 23105 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/master-0/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/master-0/wal
uuid: "9086e30907ac40d189c9847ea24713a2"
format_stamp: "Formatted at 2026-08-11 19:00:36 on dist-test-slave-0kls"
server_key: "45914f2ce1590d12e4d4a230fe45c385"
server_key_iv: "534b2ef3186e8d3a3c0a803ce88dfff9"
server_key_version: "encryptionkey@0"
I20260811 19:00:37.500440 23105 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:00:37.519186 23105 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:00:37.522867 23105 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:00:37.523098 23105 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:00:37.530895 23105 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.62:43557
I20260811 19:00:37.530912 23176 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.62:43557 every 8 connection(s)
I20260811 19:00:37.532140 23105 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
I20260811 19:00:37.536942 23177 sys_catalog.cc:263] Verifying existing consensus state
I20260811 19:00:37.536978 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 23105
I20260811 19:00:37.537492 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.1:42857
--local_ip_for_outbound_sockets=127.13.109.1
--tserver_master_addrs=127.13.109.62:43557
--webserver_port=44183
--webserver_interface=127.13.109.1
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.1.keytab
--principal=kudu/127.13.109.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:36873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.13.109.1
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/krb5kdc/oryx_127.13.109.1.keytab with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 19:00:37.539181 23177 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9086e30907ac40d189c9847ea24713a2: Bootstrap starting.
I20260811 19:00:37.564652 23177 log.cc:824] T 00000000000000000000000000000000 P 9086e30907ac40d189c9847ea24713a2: Log is configured to *not* fsync() on all Append() calls
I20260811 19:00:37.574432 23177 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9086e30907ac40d189c9847ea24713a2: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 19:00:37.574949 23177 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9086e30907ac40d189c9847ea24713a2: Bootstrap complete.
I20260811 19:00:37.581043 23177 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9086e30907ac40d189c9847ea24713a2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9086e30907ac40d189c9847ea24713a2" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 43557 } }
I20260811 19:00:37.581626 23177 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9086e30907ac40d189c9847ea24713a2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9086e30907ac40d189c9847ea24713a2, State: Initialized, Role: FOLLOWER
I20260811 19:00:37.582252 23177 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9086e30907ac40d189c9847ea24713a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9086e30907ac40d189c9847ea24713a2" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 43557 } }
I20260811 19:00:37.582424 23177 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9086e30907ac40d189c9847ea24713a2 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:00:37.582504 23177 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9086e30907ac40d189c9847ea24713a2 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:00:37.582608 23177 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9086e30907ac40d189c9847ea24713a2 [term 1 FOLLOWER]: Advancing to term 2
I20260811 19:00:37.585573 23177 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9086e30907ac40d189c9847ea24713a2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9086e30907ac40d189c9847ea24713a2" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 43557 } }
I20260811 19:00:37.586023 23177 leader_election.cc:302] T 00000000000000000000000000000000 P 9086e30907ac40d189c9847ea24713a2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9086e30907ac40d189c9847ea24713a2; no voters:
I20260811 19:00:37.586421 23177 leader_election.cc:288] T 00000000000000000000000000000000 P 9086e30907ac40d189c9847ea24713a2 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 19:00:37.586592 23181 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9086e30907ac40d189c9847ea24713a2 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 19:00:37.586908 23181 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9086e30907ac40d189c9847ea24713a2 [term 2 LEADER]: Becoming Leader. State: Replica: 9086e30907ac40d189c9847ea24713a2, State: Running, Role: LEADER
I20260811 19:00:37.587260 23181 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9086e30907ac40d189c9847ea24713a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9086e30907ac40d189c9847ea24713a2" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 43557 } }
I20260811 19:00:37.587874 23177 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9086e30907ac40d189c9847ea24713a2 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:00:37.588936 23182 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9086e30907ac40d189c9847ea24713a2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "9086e30907ac40d189c9847ea24713a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9086e30907ac40d189c9847ea24713a2" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 43557 } } }
I20260811 19:00:37.589107 23182 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9086e30907ac40d189c9847ea24713a2 [sys.catalog]: This master's current role is: LEADER
I20260811 19:00:37.588936 23183 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9086e30907ac40d189c9847ea24713a2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9086e30907ac40d189c9847ea24713a2. Latest consensus state: current_term: 2 leader_uuid: "9086e30907ac40d189c9847ea24713a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9086e30907ac40d189c9847ea24713a2" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 43557 } } }
I20260811 19:00:37.589212 23183 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9086e30907ac40d189c9847ea24713a2 [sys.catalog]: This master's current role is: LEADER
I20260811 19:00:37.589502 23191 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:00:37.593284 23191 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:00:37.594498 23191 catalog_manager.cc:1259] Loaded cluster ID: 29443ed4862f4e0f93c78996e43f60eb
I20260811 19:00:37.594592 23191 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:00:37.597111 23191 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:00:37.598042 23191 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 9086e30907ac40d189c9847ea24713a2: Loaded TSK: 0
I20260811 19:00:37.599519 23191 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 19:00:37.683374 23179 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:00:37.683820 23179 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:00:37.683907 23179 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:00:37.688885 23179 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:00:37.689009 23179 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:00:37.689167 23179 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.1
I20260811 19:00:37.696630 23179 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:36873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/krb5kdc/oryx_127.13.109.1.keytab
--principal=oryx/127.13.109.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.1:42857
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.109.1
--webserver_port=44183
--tserver_master_addrs=127.13.109.62:43557
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23179
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.1
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:00:37.698251 23179 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:00:37.699684 23179 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:00:37.713363 23209 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:37.713405 23205 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:37.713426 23206 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:37.714249 23179 server_base.cc:1034] running on GCE node
I20260811 19:00:37.714793 23179 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:00:37.715727 23179 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:00:37.716949 23179 hybrid_clock.cc:648] HybridClock initialized: now 1786474837716915 us; error 67 us; skew 500 ppm
Aug 11 19:00:37 dist-test-slave-0kls krb5kdc[22547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474837, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.109.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:00:37.720623 23179 init.cc:377] Logged in from keytab as oryx/127.13.109.1@KRBTEST.COM (short username oryx)
I20260811 19:00:37.722100 23179 webserver.cc:460] Webserver started at http://127.13.109.1:44183/ using document root <none> and password file <none>
I20260811 19:00:37.722873 23179 fs_manager.cc:359] Metadata directory not provided
I20260811 19:00:37.722981 23179 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:00:37.728416 23179 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 19:00:37.731668 23217 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:37.733280 23179 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 19:00:37.733448 23179 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/ts-0/wal
uuid: "b325635d075b4d769880d77492ccf776"
format_stamp: "Formatted at 2026-08-11 19:00:36 on dist-test-slave-0kls"
server_key: "db4ff824b6059db2eb5b06580d1527b6"
server_key_iv: "cf3f3205ce659e880787d6bec0103bde"
server_key_version: "encryptionkey@0"
I20260811 19:00:37.734189 23179 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:00:37.752871 23179 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:00:37.756949 23179 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:00:37.757205 23179 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:00:37.757999 23179 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:00:37.759073 23179 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:00:37.759128 23179 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:37.759207 23179 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:00:37.759239 23179 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:37.769445 23179 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.1:42857
I20260811 19:00:37.769470 23341 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.1:42857 every 8 connection(s)
I20260811 19:00:37.770661 23179 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
I20260811 19:00:37.778160 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 23179
I20260811 19:00:37.778680 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.2:35713
--local_ip_for_outbound_sockets=127.13.109.2
--tserver_master_addrs=127.13.109.62:43557
--webserver_port=44653
--webserver_interface=127.13.109.2
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.2.keytab
--principal=kudu/127.13.109.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:36873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.13.109.2
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/krb5kdc/oryx_127.13.109.2.keytab with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 19:00:37 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.1@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:00:37 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.1@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:00:37.787529 23346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:37.773606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:43557 (local address 127.13.109.1:59187)
0811 19:00:37.774317 (+ 711us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:37.774383 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:37.775705 (+ 1322us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:37.777006 (+ 1301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:37.777014 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:37.777516 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:37.778213 (+ 697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:37.778237 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:37.779804 (+ 1567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:37.779811 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:37.781050 (+ 1239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:37.781063 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:37.782336 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:37.782344 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:37.782587 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:37.783318 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:37.783347 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:37.786374 (+ 3027us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.109.62:43557: Server oryx/127.13.109.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":208,"threads_started":1}
W20260811 19:00:37.787567 23345 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:00:37.773592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.1:59187 (local address 127.13.109.62:43557)
0811 19:00:37.774251 (+ 659us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:37.774279 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:37.774515 (+ 236us) server_negotiation.cc:407] Connection header received
0811 19:00:37.776197 (+ 1682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:37.776208 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:37.776545 (+ 337us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:37.776910 (+ 365us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:37.778455 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:37.779599 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:37.781223 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:37.782124 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:37.782289 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:37.786650 (+ 4361us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.109.1:59187: BlockingRecv error: recv got EOF from 127.13.109.1:59187 (error 108)
Metrics: {"server-negotiator.queue_time_us":438,"thread_start_us":185,"threads_started":1}
W20260811 19:00:37.788398 23342 heartbeater.cc:644] Failed to heartbeat to 127.13.109.62:43557 (0 consecutive failures): Not authorized: Failed to ping master at 127.13.109.62:43557: Client connection negotiation failed: client connection to 127.13.109.62:43557: Server oryx/127.13.109.62@KRBTEST.COM not found in Kerberos database
Aug 11 19:00:37 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.1@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:00:37 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.1@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:00:37.795485 23346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:37.788954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:43557 (local address 127.13.109.1:37995)
0811 19:00:37.789306 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:37.789322 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:37.789431 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:37.790182 (+ 751us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:37.790216 (+ 34us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:37.790454 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:37.790577 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:37.790584 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:37.791410 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:37.791415 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:37.791989 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:37.791998 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:37.792496 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:37.792500 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:37.792597 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:37.793191 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:37.793205 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:37.795318 (+ 2113us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.109.62:43557: Server oryx/127.13.109.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":261}
W20260811 19:00:37.795922 23345 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:00:37.789275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.1:37995 (local address 127.13.109.62:43557)
0811 19:00:37.789860 (+ 585us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:37.789866 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:37.789888 (+ 22us) server_negotiation.cc:407] Connection header received
0811 19:00:37.789942 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:37.789945 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:37.790029 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:37.790141 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:37.790727 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:37.791279 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:37.792130 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:37.792370 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:37.792486 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:37.795499 (+ 3013us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.109.1:37995: BlockingRecv error: recv got EOF from 127.13.109.1:37995 (error 108)
Metrics: {"server-negotiator.queue_time_us":370}
Aug 11 19:00:37 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.1@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:00:37 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.1@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:00:37.802426 23346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:37.796093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:43557 (local address 127.13.109.1:57591)
0811 19:00:37.796268 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:37.796282 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:37.796360 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:37.796627 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:37.796636 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:37.796821 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:37.796928 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:37.796934 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:37.797699 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:37.797702 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:37.798655 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:37.798669 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:37.799206 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:37.799210 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:37.799304 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:37.799886 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:37.799902 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:37.802152 (+ 2250us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.109.62:43557: Server oryx/127.13.109.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":105}
W20260811 19:00:37.802578 23345 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:00:37.796300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.1:57591 (local address 127.13.109.62:43557)
0811 19:00:37.796420 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:37.796423 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:37.796435 (+ 12us) server_negotiation.cc:407] Connection header received
0811 19:00:37.796473 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:37.796475 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:37.796517 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:37.796586 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:37.797052 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:37.797588 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:37.798826 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:37.799075 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:37.799167 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:37.802454 (+ 3287us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.109.1:57591: BlockingRecv error: recv got EOF from 127.13.109.1:57591 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260811 19:00:37.802826 23342 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 19:00:37.909477 23347 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:00:37.909804 23347 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:00:37.909925 23347 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:00:37.914150 23347 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:00:37.914247 23347 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:00:37.914332 23347 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.2
I20260811 19:00:37.919934 23347 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:36873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/krb5kdc/oryx_127.13.109.2.keytab
--principal=oryx/127.13.109.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.2:35713
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.109.2
--webserver_port=44653
--tserver_master_addrs=127.13.109.62:43557
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23347
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.2
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:00:37.921115 23347 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:00:37.922210 23347 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:00:37.932523 23355 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:37.932518 23352 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:37.932523 23353 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:37.932793 23347 server_base.cc:1034] running on GCE node
I20260811 19:00:37.933283 23347 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:00:37.934094 23347 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:00:37.935302 23347 hybrid_clock.cc:648] HybridClock initialized: now 1786474837935263 us; error 55 us; skew 500 ppm
Aug 11 19:00:37 dist-test-slave-0kls krb5kdc[22547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474837, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.109.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:00:37.938342 23347 init.cc:377] Logged in from keytab as oryx/127.13.109.2@KRBTEST.COM (short username oryx)
I20260811 19:00:37.939493 23347 webserver.cc:460] Webserver started at http://127.13.109.2:44653/ using document root <none> and password file <none>
I20260811 19:00:37.940128 23347 fs_manager.cc:359] Metadata directory not provided
I20260811 19:00:37.940251 23347 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:00:37.944413 23347 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 19:00:37.946633 23362 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:37.947943 23347 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 19:00:37.948102 23347 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/ts-1/wal
uuid: "fb39a76844e4427c98abd64284967187"
format_stamp: "Formatted at 2026-08-11 19:00:36 on dist-test-slave-0kls"
server_key: "6e78b7e7034b3464c901e6d7c74c6d4e"
server_key_iv: "ee5b4d4f71f2e8834eea830347cd7ced"
server_key_version: "encryptionkey@0"
I20260811 19:00:37.948670 23347 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:00:37.977121 23347 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:00:37.980202 23347 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:00:37.980422 23347 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:00:37.981072 23347 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:00:37.981973 23347 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:00:37.982045 23347 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:37.982117 23347 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:00:37.982164 23347 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:37.991750 23347 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.2:35713
I20260811 19:00:37.991775 23487 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.2:35713 every 8 connection(s)
I20260811 19:00:37.992842 23347 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
I20260811 19:00:37.998399 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 23347
I20260811 19:00:37.998950 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.3:42513
--local_ip_for_outbound_sockets=127.13.109.3
--tserver_master_addrs=127.13.109.62:43557
--webserver_port=38985
--webserver_interface=127.13.109.3
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.3.keytab
--principal=kudu/127.13.109.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:36873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.13.109.3
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/krb5kdc/oryx_127.13.109.3.keytab with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 19:00:37 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.2@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:00:38 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.2@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
W20260811 19:00:38.007838 23345 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:00:37.994980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.2:41751 (local address 127.13.109.62:43557)
0811 19:00:37.995139 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:37.995143 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:37.996157 (+ 1014us) server_negotiation.cc:407] Connection header received
0811 19:00:37.997623 (+ 1466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:37.997627 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:37.997731 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:37.997871 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:37.999619 (+ 1748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:38.000172 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:38.001632 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:38.002009 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:38.002142 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:38.007653 (+ 5511us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.109.2:41751: BlockingRecv error: recv got EOF from 127.13.109.2:41751 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 19:00:38.009150 23490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:37.995407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:43557 (local address 127.13.109.2:41751)
0811 19:00:37.995941 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:37.995992 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:37.997331 (+ 1339us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:37.998102 (+ 771us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:37.998114 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:37.998654 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:37.999430 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:37.999457 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:38.000330 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:38.000336 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:38.001469 (+ 1133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:38.001481 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:38.002225 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:38.002230 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:38.002410 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:38.003149 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:38.003176 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:38.008054 (+ 4878us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.109.62:43557: Server oryx/127.13.109.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":135,"threads_started":1}
W20260811 19:00:38.010146 23488 heartbeater.cc:644] Failed to heartbeat to 127.13.109.62:43557 (0 consecutive failures): Not authorized: Failed to ping master at 127.13.109.62:43557: Client connection negotiation failed: client connection to 127.13.109.62:43557: Server oryx/127.13.109.62@KRBTEST.COM not found in Kerberos database
Aug 11 19:00:38 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.2@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:00:38 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.2@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:00:38.017664 23490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:38.010798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:43557 (local address 127.13.109.2:53291)
0811 19:00:38.011557 (+ 759us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:38.011574 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:38.011689 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:38.012060 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:38.012063 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:38.012324 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:38.012458 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:38.012465 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:38.013863 (+ 1398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:38.013866 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:38.014434 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:38.014442 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:38.015065 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:38.015070 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:38.015164 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:38.015710 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:38.015722 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:38.017550 (+ 1828us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.109.62:43557: Server oryx/127.13.109.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":647}
W20260811 19:00:38.018180 23345 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:00:38.011144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.2:53291 (local address 127.13.109.62:43557)
0811 19:00:38.011290 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:38.011295 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:38.011657 (+ 362us) server_negotiation.cc:407] Connection header received
0811 19:00:38.011840 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:38.011843 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:38.011932 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:38.012012 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:38.012739 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:38.013703 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:38.014551 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:38.014906 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:38.015034 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:38.017721 (+ 2687us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.109.2:53291: BlockingRecv error: recv got EOF from 127.13.109.2:53291 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 11 19:00:38 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.2@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:00:38 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.2@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:00:38.024142 23490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:38.018524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:43557 (local address 127.13.109.2:46049)
0811 19:00:38.018656 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:38.018668 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:38.018741 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:38.019192 (+ 451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:38.019194 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:38.019329 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:38.019406 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:38.019411 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:38.020503 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:38.020506 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:38.021017 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:38.021025 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:38.021635 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:38.021638 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:38.021720 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:38.022272 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:38.022286 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:38.024037 (+ 1751us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.109.62:43557: Server oryx/127.13.109.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":58}
W20260811 19:00:38.024446 23488 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 19:00:38.024684 23345 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:00:38.018712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.2:46049 (local address 127.13.109.62:43557)
0811 19:00:38.018956 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:38.018960 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:38.018974 (+ 14us) server_negotiation.cc:407] Connection header received
0811 19:00:38.019013 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:38.019016 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:38.019073 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:38.019158 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:38.019529 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:38.020389 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:38.021153 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:38.021495 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:38.021625 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:38.024220 (+ 2595us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.109.2:46049: BlockingRecv error: recv got EOF from 127.13.109.2:46049 (error 108)
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":82,"threads_started":1}
W20260811 19:00:38.130261 23491 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:00:38.130627 23491 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:00:38.130682 23491 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:00:38.135352 23491 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:00:38.135492 23491 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:00:38.135566 23491 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.3
I20260811 19:00:38.142501 23491 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:36873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/krb5kdc/oryx_127.13.109.3.keytab
--principal=oryx/127.13.109.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.3:42513
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.109.3
--webserver_port=38985
--tserver_master_addrs=127.13.109.62:43557
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23491
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.3
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:00:38.143838 23491 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:00:38.145004 23491 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:00:38.157558 23499 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:38.157552 23500 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:38.157899 23491 server_base.cc:1034] running on GCE node
W20260811 19:00:38.157644 23502 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:38.158704 23491 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:00:38.159694 23491 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:00:38.160949 23491 hybrid_clock.cc:648] HybridClock initialized: now 1786474838160909 us; error 69 us; skew 500 ppm
Aug 11 19:00:38 dist-test-slave-0kls krb5kdc[22547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474838, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.109.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:00:38.165248 23491 init.cc:377] Logged in from keytab as oryx/127.13.109.3@KRBTEST.COM (short username oryx)
I20260811 19:00:38.166687 23491 webserver.cc:460] Webserver started at http://127.13.109.3:38985/ using document root <none> and password file <none>
I20260811 19:00:38.167474 23491 fs_manager.cc:359] Metadata directory not provided
I20260811 19:00:38.167660 23491 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:00:38.175558 23491 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 19:00:38.179294 23512 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:38.181493 23491 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260811 19:00:38.181723 23491 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/ts-2/wal
uuid: "66a7b20a76754d5d913d2c0572d763bb"
format_stamp: "Formatted at 2026-08-11 19:00:37 on dist-test-slave-0kls"
server_key: "6c7dccb0076ef8fdd3f813aaba0bdc10"
server_key_iv: "7b37c608804df29569c60d4f9d8d35d8"
server_key_version: "encryptionkey@0"
I20260811 19:00:38.182637 23491 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:00:38.207288 23491 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:00:38.211597 23491 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:00:38.211901 23491 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:00:38.212725 23491 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:00:38.213974 23491 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:00:38.214064 23491 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:38.214155 23491 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:00:38.214205 23491 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:38.226519 23491 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.3:42513
I20260811 19:00:38.226550 23639 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.3:42513 every 8 connection(s)
I20260811 19:00:38.227917 23491 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
I20260811 19:00:38.234155 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 23491
Aug 11 19:00:38 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.3@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:00:38 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.3@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
W20260811 19:00:38.243237 23345 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:00:38.230452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.3:59395 (local address 127.13.109.62:43557)
0811 19:00:38.230663 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:38.230669 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:38.231916 (+ 1247us) server_negotiation.cc:407] Connection header received
0811 19:00:38.233418 (+ 1502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:38.233422 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:38.233484 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:38.233575 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:38.235339 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:38.236181 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:38.238160 (+ 1979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:38.238426 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:38.238491 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:38.243024 (+ 4533us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.109.3:59395: BlockingRecv error: recv got EOF from 127.13.109.3:59395 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 19:00:38.243973 23643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:38.231032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:43557 (local address 127.13.109.3:59395)
0811 19:00:38.231695 (+ 663us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:38.231754 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:38.233111 (+ 1357us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:38.233911 (+ 800us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:38.233920 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:38.234447 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:38.235080 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:38.235097 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:38.236370 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:38.236373 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:38.237967 (+ 1594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:38.237985 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:38.238729 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:38.238733 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:38.238959 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:38.239726 (+ 767us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:38.239760 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:38.242667 (+ 2907us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.109.62:43557: Server oryx/127.13.109.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":146,"threads_started":1}
W20260811 19:00:38.244983 23641 heartbeater.cc:644] Failed to heartbeat to 127.13.109.62:43557 (0 consecutive failures): Not authorized: Failed to ping master at 127.13.109.62:43557: Client connection negotiation failed: client connection to 127.13.109.62:43557: Server oryx/127.13.109.62@KRBTEST.COM not found in Kerberos database
I20260811 19:00:38.248282 23493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:38.235485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40216 (local address 127.13.109.62:43557)
0811 19:00:38.235631 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:38.235649 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:38.235663 (+ 14us) server_negotiation.cc:407] Connection header received
0811 19:00:38.235701 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:38.235703 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:38.235746 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:38.235839 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:38.236825 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:38.237664 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:38.238967 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:38.239354 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:38.239441 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:38.241363 (+ 1922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:38.241371 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:38.241393 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:38.241429 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:38.244194 (+ 2765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:38.245143 (+ 949us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:38.245158 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:38.245171 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:38.245443 (+ 272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:38.246249 (+ 806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:38.246254 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:38.246256 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:38.246887 (+ 631us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:38.247209 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:38.247652 (+ 443us) server_negotiation.cc:299] Negotiation successful
0811 19:00:38.248068 (+ 416us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
Aug 11 19:00:38 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.3@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:00:38 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.3@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:00:38.253705 23643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:38.245862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:43557 (local address 127.13.109.3:39537)
0811 19:00:38.246028 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:38.246153 (+ 125us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:38.246322 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:38.246651 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:38.246654 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:38.246913 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:38.247071 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:38.247078 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:38.248097 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:38.248111 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:38.248973 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:38.248990 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:38.249743 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:38.249746 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:38.249955 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:38.250677 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:38.250695 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:38.253560 (+ 2865us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.109.62:43557: Server oryx/127.13.109.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":58}
W20260811 19:00:38.254007 23345 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:00:38.246012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.3:39537 (local address 127.13.109.62:43557)
0811 19:00:38.246299 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:38.246304 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:38.246323 (+ 19us) server_negotiation.cc:407] Connection header received
0811 19:00:38.246457 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:38.246461 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:38.246521 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:38.246627 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:38.247215 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:38.247897 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:38.249153 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:38.249550 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:38.249717 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:38.253867 (+ 4150us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.109.3:39537: BlockingRecv error: recv got EOF from 127.13.109.3:39537 (error 108)
Metrics: {"server-negotiator.queue_time_us":141}
Aug 11 19:00:38 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.3@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:00:38 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.3@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:00:38.260905 23643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:38.254612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:43557 (local address 127.13.109.3:40245)
0811 19:00:38.254761 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:38.254781 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:38.254895 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:38.255175 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:38.255177 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:38.255352 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:38.255471 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:38.255476 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:38.256339 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:38.256342 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:38.256913 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:38.256922 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:38.257444 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:38.257447 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:38.257536 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:38.258218 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:38.258233 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:38.260727 (+ 2494us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.109.62:43557: Server oryx/127.13.109.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":52}
W20260811 19:00:38.261052 23345 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:00:38.254668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.3:40245 (local address 127.13.109.62:43557)
0811 19:00:38.254868 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:38.254872 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:38.254889 (+ 17us) server_negotiation.cc:407] Connection header received
0811 19:00:38.254998 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:38.255000 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:38.255051 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:38.255134 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:38.255620 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:38.256213 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:38.257047 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:38.257293 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:38.257390 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:38.260925 (+ 3535us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.109.3:40245: BlockingRecv error: recv got EOF from 127.13.109.3:40245 (error 108)
Metrics: {"server-negotiator.queue_time_us":115}
W20260811 19:00:38.261430 23641 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 11 19:00:38 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.1@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:00:38 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.1@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:00:38.811554 23645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:38.803754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:43557 (local address 127.13.109.1:37463)
0811 19:00:38.804136 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:38.804159 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:38.804315 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:38.804682 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:38.804685 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:38.804971 (+ 286us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:38.805162 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:38.805173 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:38.806283 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:38.806287 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:38.807002 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:38.807014 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:38.807648 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:38.807653 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:38.807803 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:38.808531 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:38.808548 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:38.811405 (+ 2857us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.109.62:43557: Server oryx/127.13.109.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":90,"threads_started":1}
W20260811 19:00:38.811841 23646 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:00:38.803847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.1:37463 (local address 127.13.109.62:43557)
0811 19:00:38.804217 (+ 370us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:38.804221 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:38.804239 (+ 18us) server_negotiation.cc:407] Connection header received
0811 19:00:38.804462 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:38.804466 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:38.804529 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:38.804635 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:38.805299 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:38.806133 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:38.807138 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:38.807399 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:38.807496 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:38.811636 (+ 4140us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.109.1:37463: BlockingRecv error: recv got EOF from 127.13.109.1:37463 (error 108)
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":149,"threads_started":1}
Aug 11 19:00:39 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.2@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:00:39 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.2@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:00:39.031291 23648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:39.025269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:43557 (local address 127.13.109.2:54651)
0811 19:00:39.025583 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:39.025600 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:39.025713 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:39.026085 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:39.026088 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:39.026338 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:39.026488 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:39.026512 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:39.027401 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:39.027404 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:39.027987 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:39.027996 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:39.028419 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:39.028422 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:39.028483 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:39.029078 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:39.029101 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:39.031076 (+ 1975us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.109.62:43557: Server oryx/127.13.109.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":89,"threads_started":1}
W20260811 19:00:39.031337 23646 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:00:39.025450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.2:54651 (local address 127.13.109.62:43557)
0811 19:00:39.025618 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:39.025621 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:39.025664 (+ 43us) server_negotiation.cc:407] Connection header received
0811 19:00:39.025837 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:39.025840 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:39.025900 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:39.026009 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:39.026647 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:39.027294 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:39.028093 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:39.028316 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:39.028394 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:39.031226 (+ 2832us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.109.2:54651: BlockingRecv error: recv got EOF from 127.13.109.2:54651 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 11 19:00:39 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.3@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:00:39 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.3@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:00:39.268657 23649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:39.262328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:43557 (local address 127.13.109.3:35791)
0811 19:00:39.262648 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:39.262664 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:39.262795 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:39.263061 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:39.263063 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:39.263277 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:39.263405 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:39.263411 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:39.264230 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:39.264232 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:39.264825 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:39.264835 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:39.265379 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:39.265382 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:39.265458 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:39.266169 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:39.266212 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:39.268442 (+ 2230us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.109.62:43557: Server oryx/127.13.109.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":82,"threads_started":1}
W20260811 19:00:39.268851 23646 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:00:39.262477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.3:35791 (local address 127.13.109.62:43557)
0811 19:00:39.262678 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:39.262682 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:39.262821 (+ 139us) server_negotiation.cc:407] Connection header received
0811 19:00:39.262881 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:39.262883 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:39.262939 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:39.263023 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:39.263520 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:39.264098 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:39.264972 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:39.265214 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:39.265312 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:39.268679 (+ 3367us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.109.3:35791: BlockingRecv error: recv got EOF from 127.13.109.3:35791 (error 108)
Metrics: {"server-negotiator.queue_time_us":96}
Aug 11 19:00:39 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.1@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:00:39 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.1@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:00:39.825717 23651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:39.812763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:43557 (local address 127.13.109.1:40541)
0811 19:00:39.813075 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:39.813095 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:39.813199 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:39.813540 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:39.813542 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:39.813754 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:39.813941 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:39.813949 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:39.814793 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:39.814796 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:39.815410 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:39.815420 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:39.816096 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:39.816103 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:39.816212 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:39.823384 (+ 7172us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:39.823405 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:39.825518 (+ 2113us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.109.62:43557: Server oryx/127.13.109.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":110,"threads_started":1}
W20260811 19:00:39.825896 23652 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:00:39.812883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.1:40541 (local address 127.13.109.62:43557)
0811 19:00:39.813195 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:39.813198 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:39.813217 (+ 19us) server_negotiation.cc:407] Connection header received
0811 19:00:39.813264 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:39.813266 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:39.813310 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:39.813440 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:39.814085 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:39.814650 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:39.815583 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:39.815917 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:39.816015 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:39.825727 (+ 9712us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.109.1:40541: BlockingRecv error: recv got EOF from 127.13.109.1:40541 (error 108)
Metrics: {"server-negotiator.queue_time_us":234,"thread_start_us":154,"threads_started":1}
Aug 11 19:00:40 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.2@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:00:40 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.2@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:00:40.038513 23653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:40.032316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:43557 (local address 127.13.109.2:51581)
0811 19:00:40.032687 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:40.032703 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:40.032823 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:40.033088 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:40.033090 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:40.033282 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:40.033391 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:40.033397 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:40.034341 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:40.034346 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:40.035158 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:40.035173 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:40.035655 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:40.035657 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:40.035723 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:40.036240 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:40.036252 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:40.038331 (+ 2079us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.109.62:43557: Server oryx/127.13.109.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":100,"threads_started":1}
W20260811 19:00:40.038681 23652 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:00:40.032427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.2:51581 (local address 127.13.109.62:43557)
0811 19:00:40.032665 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:40.032669 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:40.032782 (+ 113us) server_negotiation.cc:407] Connection header received
0811 19:00:40.032912 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:40.032914 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:40.032962 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:40.033038 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:40.033523 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:40.034207 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:40.035299 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:40.035545 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:40.035639 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:40.038546 (+ 2907us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.109.2:51581: BlockingRecv error: recv got EOF from 127.13.109.2:51581 (error 108)
Metrics: {"server-negotiator.queue_time_us":115}
Aug 11 19:00:40 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.3@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:00:40 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.3@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:00:40.276238 23654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:40.269866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:43557 (local address 127.13.109.3:54095)
0811 19:00:40.270213 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:40.270228 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:40.270330 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:40.270738 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:40.270741 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:40.270934 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:40.271043 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:40.271049 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:40.271875 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:40.271879 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:40.272458 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:40.272467 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:40.272955 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:40.272958 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:40.273030 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:40.273727 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:40.273743 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:40.276069 (+ 2326us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.109.62:43557: Server oryx/127.13.109.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":128,"threads_started":1}
W20260811 19:00:40.276367 23652 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:00:40.270111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.3:54095 (local address 127.13.109.62:43557)
0811 19:00:40.270274 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:40.270277 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:40.270293 (+ 16us) server_negotiation.cc:407] Connection header received
0811 19:00:40.270502 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:40.270507 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:40.270577 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:40.270668 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:40.271208 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:40.271753 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:40.272578 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:40.272813 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:40.272902 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:40.276239 (+ 3337us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.109.3:54095: BlockingRecv error: recv got EOF from 127.13.109.3:54095 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 11 19:00:40 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.1@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:00:40 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.1@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:00:40.835705 23656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:40.826926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:43557 (local address 127.13.109.1:56201)
0811 19:00:40.827297 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:40.827313 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:40.827464 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:40.827885 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:40.827888 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:40.828098 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:40.828237 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:40.828244 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:40.829468 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:40.829472 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:40.830361 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:40.830378 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:40.831114 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:40.831119 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:40.831246 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:40.832174 (+ 928us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:40.832198 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:40.835534 (+ 3336us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.109.62:43557: Server oryx/127.13.109.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":139,"threads_started":1}
W20260811 19:00:40.836036 23657 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:00:40.827037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.1:56201 (local address 127.13.109.62:43557)
0811 19:00:40.827329 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:40.827333 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:40.827418 (+ 85us) server_negotiation.cc:407] Connection header received
0811 19:00:40.827608 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:40.827611 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:40.827670 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:40.827781 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:40.828524 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:40.829250 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:40.830570 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:40.830920 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:40.831076 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:40.835867 (+ 4791us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.109.1:56201: BlockingRecv error: recv got EOF from 127.13.109.1:56201 (error 108)
Metrics: {"server-negotiator.queue_time_us":217,"thread_start_us":116,"threads_started":1}
Aug 11 19:00:41 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.2@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:00:41 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.2@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:00:41.046459 23658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:41.039644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:43557 (local address 127.13.109.2:35875)
0811 19:00:41.040024 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:41.040041 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:41.040159 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:41.040507 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:41.040510 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:41.040726 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:41.040848 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:41.040855 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:41.041755 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:41.041759 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:41.042393 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:41.042403 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:41.043045 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:41.043049 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:41.043146 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:41.043857 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:41.043872 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:41.046257 (+ 2385us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.109.62:43557: Server oryx/127.13.109.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":154,"threads_started":1}
W20260811 19:00:41.046553 23657 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:00:41.039881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.2:35875 (local address 127.13.109.62:43557)
0811 19:00:41.040092 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:41.040095 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:41.040116 (+ 21us) server_negotiation.cc:407] Connection header received
0811 19:00:41.040266 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:41.040268 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:41.040322 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:41.040412 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:41.040988 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:41.041583 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:41.042554 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:41.042885 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:41.042992 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:41.046418 (+ 3426us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.109.2:35875: BlockingRecv error: recv got EOF from 127.13.109.2:35875 (error 108)
Metrics: {"server-negotiator.queue_time_us":99}
Aug 11 19:00:41 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.3@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:00:41 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.3@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:00:41.283862 23659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:41.277389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:43557 (local address 127.13.109.3:60309)
0811 19:00:41.277743 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:41.277761 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:41.277948 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:41.278239 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:41.278242 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:41.278460 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:41.278557 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:41.278563 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:41.279424 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:41.279427 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:41.280289 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:41.280302 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:41.280885 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:41.280889 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:41.281035 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:41.281716 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:41.281730 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:41.283732 (+ 2002us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.109.62:43557: Server oryx/127.13.109.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":100,"threads_started":1}
W20260811 19:00:41.284060 23657 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:00:41.277505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.3:60309 (local address 127.13.109.62:43557)
0811 19:00:41.277723 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:41.277728 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:41.277879 (+ 151us) server_negotiation.cc:407] Connection header received
0811 19:00:41.278055 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:41.278058 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:41.278115 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:41.278209 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:41.278696 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:41.279256 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:41.280417 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:41.280739 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:41.280851 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:41.283889 (+ 3038us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.109.3:60309: BlockingRecv error: recv got EOF from 127.13.109.3:60309 (error 108)
Metrics: {"server-negotiator.queue_time_us":98}
Aug 11 19:00:41 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.1@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:00:41 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.1@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:00:41.842378 23660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:41.836662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:43557 (local address 127.13.109.1:34695)
0811 19:00:41.836973 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:41.836986 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:41.837099 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:41.837390 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:41.837393 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:41.837611 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:41.837719 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:41.837724 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:41.838659 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:41.838662 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:41.839306 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:41.839316 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:41.839768 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:41.839770 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:41.839834 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:41.840367 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:41.840380 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:41.842250 (+ 1870us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.109.62:43557: Server oryx/127.13.109.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":120,"threads_started":1}
W20260811 19:00:41.842571 23661 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:00:41.836773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.1:34695 (local address 127.13.109.62:43557)
0811 19:00:41.837094 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:41.837098 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:41.837114 (+ 16us) server_negotiation.cc:407] Connection header received
0811 19:00:41.837177 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:41.837180 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:41.837224 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:41.837315 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:41.837916 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:41.838540 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:41.839416 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:41.839647 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:41.839748 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:41.842416 (+ 2668us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.109.1:34695: BlockingRecv error: recv got EOF from 127.13.109.1:34695 (error 108)
Metrics: {"server-negotiator.queue_time_us":225,"thread_start_us":129,"threads_started":1}
Aug 11 19:00:42 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.2@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:00:42 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.2@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:00:42.053435 23662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:42.047441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:43557 (local address 127.13.109.2:39887)
0811 19:00:42.047913 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:42.047927 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:42.048085 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:42.048363 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:42.048366 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:42.048636 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:42.048734 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:42.048740 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:42.049506 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:42.049513 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:42.050101 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:42.050111 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:42.050579 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:42.050583 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:42.050673 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:42.051192 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:42.051203 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:42.053255 (+ 2052us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.109.62:43557: Server oryx/127.13.109.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":91,"threads_started":1}
W20260811 19:00:42.053498 23661 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:00:42.047518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.2:39887 (local address 127.13.109.62:43557)
0811 19:00:42.047707 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:42.047711 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:42.048021 (+ 310us) server_negotiation.cc:407] Connection header received
0811 19:00:42.048189 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:42.048193 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:42.048249 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:42.048340 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:42.048863 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:42.049390 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:42.050224 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:42.050451 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:42.050571 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:42.053388 (+ 2817us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.109.2:39887: BlockingRecv error: recv got EOF from 127.13.109.2:39887 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
Aug 11 19:00:42 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.3@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:00:42 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.3@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:00:42.290822 23663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:42.284866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:43557 (local address 127.13.109.3:36747)
0811 19:00:42.285158 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:42.285173 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:42.285335 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:42.285701 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:42.285704 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:42.285919 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:42.286038 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:42.286044 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:42.286771 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:42.286773 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:42.287331 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:42.287338 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:42.287817 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:42.287819 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:42.287879 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:42.288507 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:42.288520 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:42.290702 (+ 2182us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.109.62:43557: Server oryx/127.13.109.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":85,"threads_started":1}
W20260811 19:00:42.290962 23661 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:00:42.284955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.3:36747 (local address 127.13.109.62:43557)
0811 19:00:42.285096 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:42.285101 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:42.285279 (+ 178us) server_negotiation.cc:407] Connection header received
0811 19:00:42.285453 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:42.285456 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:42.285557 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:42.285645 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:42.286145 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:42.286666 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:42.287433 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:42.287716 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:42.287797 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:42.290842 (+ 3045us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.109.3:36747: BlockingRecv error: recv got EOF from 127.13.109.3:36747 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 11 19:00:42 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.1@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:00:42 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.1@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:00:42.848941 23664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:42.843413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:43557 (local address 127.13.109.1:54877)
0811 19:00:42.843686 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:42.843705 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:42.843824 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:42.844066 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:42.844068 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:42.844264 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:42.844373 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:42.844379 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:42.845163 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:42.845166 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:42.845761 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:42.845769 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:42.846309 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:42.846313 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:42.846425 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:42.846972 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:42.846984 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:42.848761 (+ 1777us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.109.62:43557: Server oryx/127.13.109.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":68,"threads_started":1}
W20260811 19:00:42.849054 23665 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:00:42.843561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.1:54877 (local address 127.13.109.62:43557)
0811 19:00:42.843843 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:42.843847 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:42.843862 (+ 15us) server_negotiation.cc:407] Connection header received
0811 19:00:42.843899 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:42.843901 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:42.843939 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:42.844013 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:42.844478 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:42.845045 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:42.845934 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:42.846186 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:42.846306 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:42.848898 (+ 2592us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.109.1:54877: BlockingRecv error: recv got EOF from 127.13.109.1:54877 (error 108)
Metrics: {"server-negotiator.queue_time_us":214,"thread_start_us":105,"threads_started":1}
Aug 11 19:00:43 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.2@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:00:43 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.2@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:00:43.059549 23666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:43.054309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:43557 (local address 127.13.109.2:34887)
0811 19:00:43.054574 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:43.054587 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:43.054680 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:43.054912 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:43.054914 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:43.055118 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:43.055218 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:43.055223 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:43.055917 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:43.055919 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:43.056474 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:43.056482 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:43.056881 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:43.056884 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:43.056942 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:43.057445 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:43.057456 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:43.059376 (+ 1920us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.109.62:43557: Server oryx/127.13.109.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":98,"threads_started":1}
W20260811 19:00:43.059593 23665 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:00:43.054450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.2:34887 (local address 127.13.109.62:43557)
0811 19:00:43.054585 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:43.054588 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:43.054643 (+ 55us) server_negotiation.cc:407] Connection header received
0811 19:00:43.054764 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:43.054766 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:43.054810 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:43.054876 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:43.055320 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:43.055824 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:43.056577 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:43.056786 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:43.056859 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:43.059494 (+ 2635us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.109.2:34887: BlockingRecv error: recv got EOF from 127.13.109.2:34887 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 11 19:00:43 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.3@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:00:43 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.3@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:00:43.297261 23667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:43.291718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:43557 (local address 127.13.109.3:37539)
0811 19:00:43.291972 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:43.291985 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:43.292075 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:43.292303 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:43.292305 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:43.292480 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:43.292573 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:43.292578 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:43.293302 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:43.293305 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:43.293973 (+ 668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:43.293987 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:43.294471 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:43.294474 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:43.294564 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:43.295032 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:43.295045 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:43.297080 (+ 2035us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.109.62:43557: Server oryx/127.13.109.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":90,"threads_started":1}
W20260811 19:00:43.297315 23665 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:00:43.291810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.3:37539 (local address 127.13.109.62:43557)
0811 19:00:43.291943 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:43.291946 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:43.292049 (+ 103us) server_negotiation.cc:407] Connection header received
0811 19:00:43.292154 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:43.292156 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:43.292201 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:43.292267 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:43.292676 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:43.293185 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:43.294125 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:43.294355 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:43.294465 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:43.297206 (+ 2741us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.109.3:37539: BlockingRecv error: recv got EOF from 127.13.109.3:37539 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 11 19:00:43 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.1@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:00:43 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.1@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:00:43.855912 23668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:43.850087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:43557 (local address 127.13.109.1:58109)
0811 19:00:43.850376 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:43.850390 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:43.850500 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:43.850754 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:43.850756 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:43.850946 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:43.851045 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:43.851050 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:43.851797 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:43.851799 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:43.852494 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:43.852504 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:43.852998 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:43.853000 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:43.853059 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:43.853695 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:43.853707 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:43.855735 (+ 2028us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.109.62:43557: Server oryx/127.13.109.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":96,"threads_started":1}
W20260811 19:00:43.856047 23669 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:00:43.850140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.1:58109 (local address 127.13.109.62:43557)
0811 19:00:43.850411 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:43.850415 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:43.850473 (+ 58us) server_negotiation.cc:407] Connection header received
0811 19:00:43.850587 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:43.850589 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:43.850638 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:43.850718 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:43.851152 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:43.851701 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:43.852634 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:43.852872 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:43.852979 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:43.855907 (+ 2928us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.109.1:58109: BlockingRecv error: recv got EOF from 127.13.109.1:58109 (error 108)
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":69,"threads_started":1}
Aug 11 19:00:44 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.2@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:00:44 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.2@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:00:44.066304 23670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:44.060440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:43557 (local address 127.13.109.2:54059)
0811 19:00:44.060766 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:44.060780 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:44.060869 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:44.061148 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:44.061151 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:44.061356 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:44.061463 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:44.061468 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:44.062556 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:44.062559 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:44.063131 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:44.063140 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:44.063571 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:44.063573 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:44.063638 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:44.064155 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:44.064166 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:44.066152 (+ 1986us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.109.62:43557: Server oryx/127.13.109.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":94,"threads_started":1}
W20260811 19:00:44.066514 23669 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:00:44.060646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.2:54059 (local address 127.13.109.62:43557)
0811 19:00:44.060806 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:44.060809 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:44.060840 (+ 31us) server_negotiation.cc:407] Connection header received
0811 19:00:44.060964 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:44.060966 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:44.061011 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:44.061109 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:44.061854 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:44.062445 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:44.063248 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:44.063462 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:44.063546 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:44.066376 (+ 2830us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.109.2:54059: BlockingRecv error: recv got EOF from 127.13.109.2:54059 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 11 19:00:44 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.3@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:00:44 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.3@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:00:44.304162 23671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:44.298347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:43557 (local address 127.13.109.3:59369)
0811 19:00:44.298663 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:44.298679 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:44.298804 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:44.299167 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:44.299170 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:44.299398 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:44.299491 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:44.299496 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:44.300276 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:44.300281 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:44.300839 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:44.300846 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:44.301323 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:44.301326 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:44.301413 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:44.302004 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:44.302025 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:44.303957 (+ 1932us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.109.62:43557: Server oryx/127.13.109.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":85,"threads_started":1}
W20260811 19:00:44.304237 23669 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:00:44.298432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.3:59369 (local address 127.13.109.62:43557)
0811 19:00:44.298582 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:44.298587 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:44.298736 (+ 149us) server_negotiation.cc:407] Connection header received
0811 19:00:44.298903 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:44.298906 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:44.298989 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:44.299109 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:44.299611 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:44.300158 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:44.300965 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:44.301196 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:44.301315 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:44.304121 (+ 2806us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.109.3:59369: BlockingRecv error: recv got EOF from 127.13.109.3:59369 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 11 19:00:44 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.1@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:00:44 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.1@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:00:44.863768 23672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:44.857151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:43557 (local address 127.13.109.1:54359)
0811 19:00:44.857479 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:44.857495 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:44.857650 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:44.858027 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:44.858030 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:44.858284 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:44.858416 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:44.858426 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:44.859421 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:44.859425 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:44.860058 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:44.860067 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:44.860566 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:44.860570 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:44.860655 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:44.861245 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:44.861257 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:44.863532 (+ 2275us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.109.62:43557: Server oryx/127.13.109.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":90,"threads_started":1}
W20260811 19:00:44.863897 23673 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:00:44.857221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.1:54359 (local address 127.13.109.62:43557)
0811 19:00:44.857483 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:44.857487 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:44.857590 (+ 103us) server_negotiation.cc:407] Connection header received
0811 19:00:44.857822 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:44.857829 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:44.857895 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:44.857988 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:44.858526 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:44.859302 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:44.860194 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:44.860442 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:44.860551 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:44.863731 (+ 3180us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.109.1:54359: BlockingRecv error: recv got EOF from 127.13.109.1:54359 (error 108)
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":83,"threads_started":1}
Aug 11 19:00:45 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.2@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:00:45 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.2@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:00:45.072803 23674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:45.067210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:43557 (local address 127.13.109.2:41831)
0811 19:00:45.067582 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:45.067598 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:45.067705 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:45.067941 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:45.067943 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:45.068153 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:45.068267 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:45.068273 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:45.069014 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:45.069017 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:45.069605 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:45.069614 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:45.070069 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:45.070072 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:45.070158 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:45.070649 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:45.070660 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:45.072608 (+ 1948us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.109.62:43557: Server oryx/127.13.109.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":107,"threads_started":1}
W20260811 19:00:45.072860 23673 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:00:45.067299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.2:41831 (local address 127.13.109.62:43557)
0811 19:00:45.067471 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:45.067476 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:45.067675 (+ 199us) server_negotiation.cc:407] Connection header received
0811 19:00:45.067791 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:45.067793 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:45.067839 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:45.067899 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:45.068376 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:45.068890 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:45.069720 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:45.069958 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:45.070037 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:45.072748 (+ 2711us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.109.2:41831: BlockingRecv error: recv got EOF from 127.13.109.2:41831 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 11 19:00:45 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.3@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:00:45 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.3@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:00:45.311034 23675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:45.305228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:43557 (local address 127.13.109.3:44073)
0811 19:00:45.305600 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:45.305613 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:45.305752 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:45.306066 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:45.306069 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:45.306297 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:45.306387 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:45.306393 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:45.307171 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:45.307175 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:45.307731 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:45.307738 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:45.308215 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:45.308219 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:45.308304 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:45.308886 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:45.308899 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:45.310848 (+ 1949us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.109.62:43557: Server oryx/127.13.109.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":98,"threads_started":1}
W20260811 19:00:45.311107 23673 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:00:45.305337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.3:44073 (local address 127.13.109.62:43557)
0811 19:00:45.305507 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:45.305511 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:45.305692 (+ 181us) server_negotiation.cc:407] Connection header received
0811 19:00:45.305906 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:45.305909 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:45.305959 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:45.306046 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:45.306506 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:45.307038 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:45.307854 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:45.308086 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:45.308199 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:45.310991 (+ 2792us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.109.3:44073: BlockingRecv error: recv got EOF from 127.13.109.3:44073 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 11 19:00:45 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.1@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:00:45 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.1@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:00:45.870709 23676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:45.864742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:43557 (local address 127.13.109.1:54013)
0811 19:00:45.865079 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:45.865093 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:45.865198 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:45.865502 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:45.865504 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:45.865704 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:45.865839 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:45.865845 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:45.866611 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:45.866614 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:45.867182 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:45.867191 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:45.867620 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:45.867623 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:45.867688 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:45.868256 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:45.868268 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:45.870511 (+ 2243us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.109.62:43557: Server oryx/127.13.109.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":102,"threads_started":1}
W20260811 19:00:45.870781 23677 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:00:45.864991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.1:54013 (local address 127.13.109.62:43557)
0811 19:00:45.865239 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:45.865242 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:45.865256 (+ 14us) server_negotiation.cc:407] Connection header received
0811 19:00:45.865302 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:45.865304 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:45.865348 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:45.865431 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:45.865947 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:45.866487 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:45.867287 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:45.867506 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:45.867584 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:45.870652 (+ 3068us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.109.1:54013: BlockingRecv error: recv got EOF from 127.13.109.1:54013 (error 108)
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":85,"threads_started":1}
Aug 11 19:00:46 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.2@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:00:46 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.2@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:00:46.079799 23678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:46.073640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:43557 (local address 127.13.109.2:40839)
0811 19:00:46.074096 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:46.074113 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:46.074283 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:46.074589 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:46.074592 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:46.074799 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:46.074894 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:46.074900 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:46.075804 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:46.075808 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:46.076366 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:46.076373 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:46.076824 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:46.076827 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:46.076910 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:46.077464 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:46.077476 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:46.079603 (+ 2127us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.109.62:43557: Server oryx/127.13.109.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":134,"threads_started":1}
W20260811 19:00:46.079861 23677 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:00:46.073835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.2:40839 (local address 127.13.109.62:43557)
0811 19:00:46.074081 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:46.074086 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:46.074216 (+ 130us) server_negotiation.cc:407] Connection header received
0811 19:00:46.074398 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:46.074406 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:46.074451 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:46.074547 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:46.075010 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:46.075687 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:46.076482 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:46.076709 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:46.076816 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:46.079750 (+ 2934us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.109.2:40839: BlockingRecv error: recv got EOF from 127.13.109.2:40839 (error 108)
Metrics: {"server-negotiator.queue_time_us":107}
Aug 11 19:00:46 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.3@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:00:46 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.3@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:00:46.318567 23679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:46.312131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:43557 (local address 127.13.109.3:58765)
0811 19:00:46.312425 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:46.312440 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:46.312551 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:46.312896 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:46.312898 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:46.313078 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:46.313179 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:46.313185 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:46.314090 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:46.314092 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:46.314665 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:46.314673 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:46.315146 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:46.315149 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:46.315267 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:46.315937 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:46.315953 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:46.318362 (+ 2409us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.109.62:43557: Server oryx/127.13.109.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":106,"threads_started":1}
W20260811 19:00:46.318642 23677 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:00:46.312194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.3:58765 (local address 127.13.109.62:43557)
0811 19:00:46.312364 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:46.312369 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:46.312522 (+ 153us) server_negotiation.cc:407] Connection header received
0811 19:00:46.312721 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:46.312724 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:46.312777 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:46.312857 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:46.313300 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:46.313971 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:46.314786 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:46.315000 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:46.315113 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:46.318519 (+ 3406us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.109.3:58765: BlockingRecv error: recv got EOF from 127.13.109.3:58765 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 11 19:00:46 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.1@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:00:46 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.1@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:00:46.877918 23681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:46.871977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:43557 (local address 127.13.109.1:37517)
0811 19:00:46.872306 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:46.872324 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:46.872462 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:46.872789 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:46.872792 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:46.873011 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:46.873135 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:46.873142 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:46.874025 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:46.874029 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:46.874595 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:46.874604 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:46.875096 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:46.875099 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:46.875180 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:46.875777 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:46.875791 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:46.877694 (+ 1903us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.109.62:43557: Server oryx/127.13.109.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":97,"threads_started":1}
W20260811 19:00:46.878002 23682 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:00:46.872050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.1:37517 (local address 127.13.109.62:43557)
0811 19:00:46.872351 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:46.872354 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:46.872394 (+ 40us) server_negotiation.cc:407] Connection header received
0811 19:00:46.872585 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:46.872588 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:46.872654 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:46.872762 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:46.873266 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:46.873905 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:46.874724 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:46.874967 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:46.875089 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:46.877874 (+ 2785us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.109.1:37517: BlockingRecv error: recv got EOF from 127.13.109.1:37517 (error 108)
Metrics: {"server-negotiator.queue_time_us":215,"thread_start_us":68,"threads_started":1}
Aug 11 19:00:47 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.2@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:00:47 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.2@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:00:47.086236 23683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:47.080755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:43557 (local address 127.13.109.2:51215)
0811 19:00:47.081045 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:47.081059 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:47.081165 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:47.081419 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:47.081421 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:47.081608 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:47.081713 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:47.081718 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:47.082479 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:47.082481 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:47.083040 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:47.083049 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:47.083475 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:47.083477 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:47.083541 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:47.084046 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:47.084058 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:47.086087 (+ 2029us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.109.62:43557: Server oryx/127.13.109.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":79,"threads_started":1}
W20260811 19:00:47.086405 23682 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:00:47.080900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.2:51215 (local address 127.13.109.62:43557)
0811 19:00:47.081083 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:47.081087 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:47.081104 (+ 17us) server_negotiation.cc:407] Connection header received
0811 19:00:47.081259 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:47.081261 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:47.081306 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:47.081375 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:47.081836 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:47.082374 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:47.083161 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:47.083373 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:47.083448 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:47.086245 (+ 2797us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.109.2:51215: BlockingRecv error: recv got EOF from 127.13.109.2:51215 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Aug 11 19:00:47 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.3@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:00:47 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.3@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:00:47.325718 23684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:47.319664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:43557 (local address 127.13.109.3:37933)
0811 19:00:47.320048 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:47.320087 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:47.320251 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:47.320545 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:47.320548 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:47.320775 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:47.320871 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:47.320877 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:47.321674 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:47.321680 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:47.322301 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:47.322311 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:47.322806 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:47.322810 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:47.322896 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:47.323468 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:47.323480 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:47.325561 (+ 2081us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.109.62:43557: Server oryx/127.13.109.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":120,"threads_started":1}
W20260811 19:00:47.325981 23682 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:00:47.319799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.3:37933 (local address 127.13.109.62:43557)
0811 19:00:47.320013 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:47.320018 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:47.320180 (+ 162us) server_negotiation.cc:407] Connection header received
0811 19:00:47.320364 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:47.320367 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:47.320425 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:47.320519 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:47.321006 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:47.321543 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:47.322434 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:47.322672 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:47.322784 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:47.325854 (+ 3070us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.109.3:37933: BlockingRecv error: recv got EOF from 127.13.109.3:37933 (error 108)
Metrics: {"server-negotiator.queue_time_us":106}
Aug 11 19:00:47 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.1@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:00:47 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.1@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:00:47.884409 23685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:47.878889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:43557 (local address 127.13.109.1:43431)
0811 19:00:47.879183 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:47.879200 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:47.879304 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:47.879553 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:47.879555 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:47.879773 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:47.879882 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:47.879888 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:47.880689 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:47.880692 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:47.881259 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:47.881268 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:47.881761 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:47.881764 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:47.881851 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:47.882398 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:47.882410 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:47.884282 (+ 1872us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.109.62:43557: Server oryx/127.13.109.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":109,"threads_started":1}
W20260811 19:00:47.884626 23686 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:00:47.879028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.1:43431 (local address 127.13.109.62:43557)
0811 19:00:47.879299 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:47.879303 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:47.879316 (+ 13us) server_negotiation.cc:407] Connection header received
0811 19:00:47.879378 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:47.879380 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:47.879421 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:47.879509 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:47.879998 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:47.880546 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:47.881404 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:47.881645 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:47.881738 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:47.884442 (+ 2704us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.109.1:43431: BlockingRecv error: recv got EOF from 127.13.109.1:43431 (error 108)
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":106,"threads_started":1}
Aug 11 19:00:48 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.2@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:00:48 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.2@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:00:48.093014 23687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:48.087250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:43557 (local address 127.13.109.2:51609)
0811 19:00:48.087557 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:48.087571 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:48.087670 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:48.087913 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:48.087915 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:48.088183 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:48.088286 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:48.088293 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:48.089264 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:48.089267 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:48.089937 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:48.089950 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:48.090429 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:48.090432 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:48.090496 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:48.090987 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:48.090999 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:48.092829 (+ 1830us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.109.62:43557: Server oryx/127.13.109.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":117,"threads_started":1}
W20260811 19:00:48.093071 23686 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:00:48.087306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.2:51609 (local address 127.13.109.62:43557)
0811 19:00:48.087460 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:48.087465 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:48.087647 (+ 182us) server_negotiation.cc:407] Connection header received
0811 19:00:48.087752 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:48.087754 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:48.087797 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:48.087884 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:48.088388 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:48.089074 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:48.090044 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:48.090324 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:48.090409 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:48.092959 (+ 2550us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.109.2:51609: BlockingRecv error: recv got EOF from 127.13.109.2:51609 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 11 19:00:48 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.3@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:00:48 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.3@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:00:48.332314 23688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:48.326604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:43557 (local address 127.13.109.3:46473)
0811 19:00:48.326956 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:48.326973 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:48.327084 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:48.327365 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:48.327369 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:48.327594 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:48.327684 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:48.327690 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:48.328494 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:48.328499 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:48.329060 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:48.329069 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:48.329564 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:48.329568 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:48.329656 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:48.330245 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:48.330257 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:48.332128 (+ 1871us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.109.62:43557: Server oryx/127.13.109.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":117,"threads_started":1}
W20260811 19:00:48.332362 23686 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:00:48.326802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.3:46473 (local address 127.13.109.62:43557)
0811 19:00:48.326956 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:48.326962 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:48.327032 (+ 70us) server_negotiation.cc:407] Connection header received
0811 19:00:48.327194 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:48.327197 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:48.327248 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:48.327342 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:48.327814 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:48.328379 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:48.329193 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:48.329429 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:48.329539 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:48.332256 (+ 2717us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.109.3:46473: BlockingRecv error: recv got EOF from 127.13.109.3:46473 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 11 19:00:48 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.1@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:00:48 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.1@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:00:48.891733 23689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:48.885392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:43557 (local address 127.13.109.1:58837)
0811 19:00:48.885695 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:48.885710 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:48.885854 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:48.886141 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:48.886143 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:48.886337 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:48.886445 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:48.886451 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:48.887438 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:48.887440 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:48.888083 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:48.888093 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:48.888630 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:48.888634 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:48.888706 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:48.889292 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:48.889304 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:48.891588 (+ 2284us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.109.62:43557: Server oryx/127.13.109.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":102,"threads_started":1}
W20260811 19:00:48.891932 23690 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:00:48.885470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.1:58837 (local address 127.13.109.62:43557)
0811 19:00:48.885735 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:48.885739 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:48.885756 (+ 17us) server_negotiation.cc:407] Connection header received
0811 19:00:48.885970 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:48.885972 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:48.886018 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:48.886102 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:48.886560 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:48.887316 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:48.888221 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:48.888449 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:48.888594 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:48.891774 (+ 3180us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.109.1:58837: BlockingRecv error: recv got EOF from 127.13.109.1:58837 (error 108)
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":93,"threads_started":1}
Aug 11 19:00:49 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.2@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:00:49 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.2@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:00:49.099599 23691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:49.094017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:43557 (local address 127.13.109.2:49095)
0811 19:00:49.094279 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:49.094292 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:49.094437 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:49.094764 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:49.094772 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:49.094962 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:49.095095 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:49.095104 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:49.095885 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:49.095889 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:49.096450 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:49.096459 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:49.096873 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:49.096875 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:49.096939 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:49.097483 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:49.097496 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:49.099407 (+ 1911us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.109.62:43557: Server oryx/127.13.109.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":83,"threads_started":1}
W20260811 19:00:49.099753 23690 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:00:49.094073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.2:49095 (local address 127.13.109.62:43557)
0811 19:00:49.094229 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:49.094234 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:49.094366 (+ 132us) server_negotiation.cc:407] Connection header received
0811 19:00:49.094543 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:49.094546 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:49.094604 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:49.094705 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:49.095219 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:49.095758 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:49.096562 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:49.096772 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:49.096851 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:49.099582 (+ 2731us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.109.2:49095: BlockingRecv error: recv got EOF from 127.13.109.2:49095 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 11 19:00:49 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.3@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:00:49 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.3@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:00:49.339529 23692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:49.333472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:43557 (local address 127.13.109.3:40379)
0811 19:00:49.333868 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:49.333883 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:49.334010 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:49.334363 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:49.334366 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:49.334600 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:49.334742 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:49.334750 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:49.335810 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:49.335814 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:49.336391 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:49.336400 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:49.336841 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:49.336843 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:49.336908 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:49.337389 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:49.337401 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:49.339404 (+ 2003us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.109.62:43557: Server oryx/127.13.109.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":112,"threads_started":1}
W20260811 19:00:49.339696 23690 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:00:49.333562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.3:40379 (local address 127.13.109.62:43557)
0811 19:00:49.333723 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:49.333729 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:49.333963 (+ 234us) server_negotiation.cc:407] Connection header received
0811 19:00:49.334136 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:49.334139 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:49.334190 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:49.334268 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:49.334932 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:49.335681 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:49.336516 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:49.336740 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:49.336820 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:49.339561 (+ 2741us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.109.3:40379: BlockingRecv error: recv got EOF from 127.13.109.3:40379 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 11 19:00:49 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.1@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:00:49 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.1@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:00:49.902737 23694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:49.892772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:43557 (local address 127.13.109.1:40859)
0811 19:00:49.893094 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:49.893107 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:49.893211 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:49.893470 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:49.893472 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:49.893672 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:49.893770 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:49.893776 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:49.894573 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:49.894576 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:49.895146 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:49.895155 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:49.895686 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:49.895690 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:49.895775 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:49.900466 (+ 4691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:49.900482 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:49.902603 (+ 2121us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.109.62:43557: Server oryx/127.13.109.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":91,"threads_started":1}
W20260811 19:00:49.902956 23695 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:00:49.892895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.1:40859 (local address 127.13.109.62:43557)
0811 19:00:49.893154 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:49.893158 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:49.893174 (+ 16us) server_negotiation.cc:407] Connection header received
0811 19:00:49.893307 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:49.893309 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:49.893356 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:49.893434 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:49.893913 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:49.894448 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:49.895281 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:49.895511 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:49.895634 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:49.902801 (+ 7167us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.109.1:40859: BlockingRecv error: recv got EOF from 127.13.109.1:40859 (error 108)
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":62,"threads_started":1}
Aug 11 19:00:50 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.2@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:00:50 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.2@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:00:50.105973 23697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:50.100469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:43557 (local address 127.13.109.2:43243)
0811 19:00:50.100858 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:50.100874 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:50.100985 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:50.101214 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:50.101216 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:50.101398 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:50.101494 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:50.101500 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:50.102319 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:50.102322 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:50.102889 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:50.102898 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:50.103326 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:50.103329 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:50.103390 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:50.103907 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:50.103919 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:50.105848 (+ 1929us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.109.62:43557: Server oryx/127.13.109.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":81,"threads_started":1}
W20260811 19:00:50.106076 23695 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:00:50.100617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.2:43243 (local address 127.13.109.62:43557)
0811 19:00:50.100765 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:50.100770 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:50.100952 (+ 182us) server_negotiation.cc:407] Connection header received
0811 19:00:50.101054 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:50.101056 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:50.101102 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:50.101182 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:50.101585 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:50.102203 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:50.102995 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:50.103224 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:50.103308 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:50.105959 (+ 2651us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.109.2:43243: BlockingRecv error: recv got EOF from 127.13.109.2:43243 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 11 19:00:50 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.3@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:00:50 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.3@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:00:50.345511 23698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:50.340304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:43557 (local address 127.13.109.3:40825)
0811 19:00:50.340587 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:50.340600 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:50.340699 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:50.340932 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:50.340934 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:50.341126 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:50.341223 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:50.341228 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:50.341996 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:50.341998 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:50.342547 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:50.342554 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:50.342952 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:50.342955 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:50.343014 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:50.343571 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:50.343583 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:50.345381 (+ 1798us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.109.62:43557: Server oryx/127.13.109.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":87,"threads_started":1}
W20260811 19:00:50.345710 23695 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:00:50.340408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.3:40825 (local address 127.13.109.62:43557)
0811 19:00:50.340582 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:50.340586 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:50.340652 (+ 66us) server_negotiation.cc:407] Connection header received
0811 19:00:50.340780 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:50.340782 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:50.340829 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:50.340896 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:50.341318 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:50.341901 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:50.342644 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:50.342858 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:50.342933 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:50.345536 (+ 2603us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.109.3:40825: BlockingRecv error: recv got EOF from 127.13.109.3:40825 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
Aug 11 19:00:50 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.1@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:00:50 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.1@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:00:50.909103 23699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:50.903606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:43557 (local address 127.13.109.1:38661)
0811 19:00:50.903900 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:50.903914 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:50.904009 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:50.904333 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:50.904335 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:50.904510 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:50.904607 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:50.904612 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:50.905361 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:50.905365 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:50.906086 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:50.906098 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:50.906607 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:50.906611 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:50.906694 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:50.907208 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:50.907220 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:50.908929 (+ 1709us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.109.62:43557: Server oryx/127.13.109.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":80,"threads_started":1}
W20260811 19:00:50.909185 23700 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:00:50.903648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.1:38661 (local address 127.13.109.62:43557)
0811 19:00:50.903950 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:50.903954 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:50.903968 (+ 14us) server_negotiation.cc:407] Connection header received
0811 19:00:50.904094 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:50.904096 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:50.904140 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:50.904250 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:50.904712 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:50.905255 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:50.906212 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:50.906488 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:50.906607 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:50.909060 (+ 2453us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.109.1:38661: BlockingRecv error: recv got EOF from 127.13.109.1:38661 (error 108)
Metrics: {"server-negotiator.queue_time_us":232,"thread_start_us":128,"threads_started":1}
Aug 11 19:00:51 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.2@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:00:51 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.2@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:00:51.111996 23701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:51.106795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:43557 (local address 127.13.109.2:42367)
0811 19:00:51.107081 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:51.107095 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:51.107216 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:51.107514 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:51.107518 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:51.107723 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:51.107827 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:51.107834 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:51.108561 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:51.108563 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:51.109112 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:51.109119 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:51.109527 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:51.109529 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:51.109591 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:51.110091 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:51.110102 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:51.111878 (+ 1776us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.109.62:43557: Server oryx/127.13.109.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":102,"threads_started":1}
W20260811 19:00:51.112187 23700 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:00:51.106913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.2:42367 (local address 127.13.109.62:43557)
0811 19:00:51.107207 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:51.107211 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:51.107225 (+ 14us) server_negotiation.cc:407] Connection header received
0811 19:00:51.107308 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:51.107310 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:51.107372 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:51.107454 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:51.107934 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:51.108463 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:51.109225 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:51.109431 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:51.109506 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:51.112015 (+ 2509us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.109.2:42367: BlockingRecv error: recv got EOF from 127.13.109.2:42367 (error 108)
Metrics: {"server-negotiator.queue_time_us":174}
Aug 11 19:00:51 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.3@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:00:51 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.3@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:00:51.351778 23702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:51.346443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:43557 (local address 127.13.109.3:42043)
0811 19:00:51.346724 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:51.346737 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:51.346837 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:51.347062 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:51.347064 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:51.347254 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:51.347350 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:51.347355 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:51.348052 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:51.348054 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:51.348625 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:51.348632 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:51.349067 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:51.349070 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:51.349140 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:51.349756 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:51.349769 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:51.351592 (+ 1823us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.109.62:43557: Server oryx/127.13.109.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":104,"threads_started":1}
W20260811 19:00:51.351866 23700 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:00:51.346506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.3:42043 (local address 127.13.109.62:43557)
0811 19:00:51.346628 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:51.346632 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:51.346799 (+ 167us) server_negotiation.cc:407] Connection header received
0811 19:00:51.346911 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:51.346913 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:51.346960 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:51.347025 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:51.347442 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:51.347960 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:51.348730 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:51.348939 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:51.349016 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:51.351729 (+ 2713us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.109.3:42043: BlockingRecv error: recv got EOF from 127.13.109.3:42043 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 11 19:00:51 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.1@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:00:51 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.1@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:00:51.915416 23703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:51.910002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:43557 (local address 127.13.109.1:40697)
0811 19:00:51.910265 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:51.910280 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:51.910378 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:51.910631 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:51.910633 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:51.910827 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:51.910928 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:51.910933 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:51.911701 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:51.911703 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:51.912243 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:51.912251 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:51.912669 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:51.912671 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:51.912742 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:51.913224 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:51.913236 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:51.915287 (+ 2051us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.109.62:43557: Server oryx/127.13.109.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":90,"threads_started":1}
W20260811 19:00:51.915632 23704 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:00:51.910065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.1:40697 (local address 127.13.109.62:43557)
0811 19:00:51.910363 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:51.910367 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:51.910382 (+ 15us) server_negotiation.cc:407] Connection header received
0811 19:00:51.910461 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:51.910463 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:51.910509 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:51.910600 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:51.911023 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:51.911598 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:51.912340 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:51.912572 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:51.912648 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:51.915488 (+ 2840us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.109.1:40697: BlockingRecv error: recv got EOF from 127.13.109.1:40697 (error 108)
Metrics: {"server-negotiator.queue_time_us":224,"thread_start_us":87,"threads_started":1}
Aug 11 19:00:52 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.2@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:00:52 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.2@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:00:52.118863 23705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:52.112891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:43557 (local address 127.13.109.2:59085)
0811 19:00:52.113229 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:52.113243 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:52.113381 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:52.113664 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:52.113667 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:52.113928 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:52.114026 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:52.114031 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:52.114822 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:52.114827 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:52.115429 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:52.115438 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:52.115907 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:52.115911 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:52.115997 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:52.116514 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:52.116526 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:52.118651 (+ 2125us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.109.62:43557: Server oryx/127.13.109.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":96,"threads_started":1}
W20260811 19:00:52.119112 23704 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:00:52.112999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.2:59085 (local address 127.13.109.62:43557)
0811 19:00:52.113171 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:52.113176 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:52.113316 (+ 140us) server_negotiation.cc:407] Connection header received
0811 19:00:52.113491 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:52.113497 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:52.113554 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:52.113640 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:52.114142 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:52.114718 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:52.115554 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:52.115788 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:52.115888 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:52.118823 (+ 2935us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.109.2:59085: BlockingRecv error: recv got EOF from 127.13.109.2:59085 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 11 19:00:52 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.3@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:00:52 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.3@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:00:52.358280 23706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:52.352731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:43557 (local address 127.13.109.3:48481)
0811 19:00:52.353033 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:52.353047 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:52.353152 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:52.353493 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:52.353495 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:52.353692 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:52.353831 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:52.353838 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:52.354676 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:52.354678 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:52.355256 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:52.355264 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:52.355693 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:52.355696 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:52.355759 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:52.356236 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:52.356247 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:52.358102 (+ 1855us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.109.62:43557: Server oryx/127.13.109.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":110,"threads_started":1}
W20260811 19:00:52.358421 23704 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:00:52.352917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.3:48481 (local address 127.13.109.62:43557)
0811 19:00:52.353049 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:52.353052 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:52.353103 (+ 51us) server_negotiation.cc:407] Connection header received
0811 19:00:52.353284 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:52.353287 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:52.353339 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:52.353418 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:52.353955 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:52.354559 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:52.355359 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:52.355571 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:52.355649 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:52.358269 (+ 2620us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.109.3:48481: BlockingRecv error: recv got EOF from 127.13.109.3:48481 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 11 19:00:52 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.1@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:00:52 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.1@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:00:52.922266 23707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:52.916607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:43557 (local address 127.13.109.1:37411)
0811 19:00:52.917015 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:52.917034 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:52.917160 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:52.917455 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:52.917458 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:52.917717 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:52.917878 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:52.917886 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:52.918619 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:52.918621 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:52.919192 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:52.919200 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:52.919601 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:52.919604 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:52.919663 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:52.920155 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:52.920167 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:52.922077 (+ 1910us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.109.62:43557: Server oryx/127.13.109.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":117,"threads_started":1}
W20260811 19:00:52.922381 23708 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:00:52.916740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.1:37411 (local address 127.13.109.62:43557)
0811 19:00:52.917008 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:52.917011 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:52.917120 (+ 109us) server_negotiation.cc:407] Connection header received
0811 19:00:52.917255 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:52.917258 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:52.917314 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:52.917418 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:52.917986 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:52.918528 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:52.919292 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:52.919505 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:52.919579 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:52.922236 (+ 2657us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.109.1:37411: BlockingRecv error: recv got EOF from 127.13.109.1:37411 (error 108)
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":93,"threads_started":1}
Aug 11 19:00:53 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.2@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:00:53 dist-test-slave-0kls krb5kdc[22547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.109.2@KRBTEST.COM for oryx/127.13.109.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:00:53.125109 23709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:53.119839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:43557 (local address 127.13.109.2:53351)
0811 19:00:53.120136 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:53.120149 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:53.120247 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:53.120479 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:53.120481 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:53.120664 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:53.120780 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:53.120786 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:53.121505 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:53.121508 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:53.122148 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:53.122158 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:53.122571 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:53.122573 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:53.122636 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:53.123136 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:53.123147 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:53.124990 (+ 1843us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.109.62:43557: Server oryx/127.13.109.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":117,"threads_started":1}
W20260811 19:00:53.125264 23708 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:00:53.119896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.2:53351 (local address 127.13.109.62:43557)
0811 19:00:53.120042 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:53.120046 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:53.120218 (+ 172us) server_negotiation.cc:407] Connection header received
0811 19:00:53.120323 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:53.120325 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:53.120369 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:53.120440 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:53.120878 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:53.121406 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:53.122261 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:53.122473 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:53.122551 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:53.125144 (+ 2593us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.109.2:53351: BlockingRecv error: recv got EOF from 127.13.109.2:53351 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 19:00:53.235360 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 23179
I20260811 19:00:53.246824 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 23347
I20260811 19:00:53.257510 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 23491
I20260811 19:00:53.266853 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 23105
2026-08-11T19:00:53Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18479 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-11T19:00:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:00:53Z Disabled control of system clock
I20260811 19:00:53.303141 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474792496492-13748-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474792496492-13748-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474792496492-13748-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.62:36033
--webserver_interface=127.13.109.62
--webserver_port=0
--builtin_ntp_servers=127.13.109.20:45441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.109.62:36033
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260811 19:00:53.422292 23713 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:00:53.422613 23713 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:00:53.422649 23713 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:00:53.426800 23713 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:00:53.426882 23713 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:00:53.426899 23713 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:00:53.426916 23713 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:00:53.426932 23713 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:00:53.431962 23713 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:45441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474792496492-13748-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474792496492-13748-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.109.62:36033
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.62:36033
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.109.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23713
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474792496492-13748-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:00:53.433202 23713 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:00:53.434727 23713 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:00:53.443857 23719 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:53.443974 23713 server_base.cc:1034] running on GCE node
W20260811 19:00:53.444173 23718 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:53.443884 23721 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:53.445039 23713 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:00:53.446345 23713 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:00:53.447559 23713 hybrid_clock.cc:648] HybridClock initialized: now 1786474853447543 us; error 28 us; skew 500 ppm
I20260811 19:00:53.449604 23713 webserver.cc:460] Webserver started at http://127.13.109.62:33497/ using document root <none> and password file <none>
I20260811 19:00:53.450282 23713 fs_manager.cc:359] Metadata directory not provided
I20260811 19:00:53.450349 23713 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:00:53.450532 23713 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:00:53.452720 23713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474792496492-13748-0/minicluster-data/master-0/data/instance:
uuid: "1fb2c37c64424c7fb0e7a17f5f454c43"
format_stamp: "Formatted at 2026-08-11 19:00:53 on dist-test-slave-0kls"
server_key: "d738208d3f6a0d094531fe5b6aed4a7c"
server_key_iv: "0a6fdc966e7d5df523ab96295038c914"
server_key_version: "encryptionkey@0"
I20260811 19:00:53.453285 23713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474792496492-13748-0/minicluster-data/master-0/wal/instance:
uuid: "1fb2c37c64424c7fb0e7a17f5f454c43"
format_stamp: "Formatted at 2026-08-11 19:00:53 on dist-test-slave-0kls"
server_key: "d738208d3f6a0d094531fe5b6aed4a7c"
server_key_iv: "0a6fdc966e7d5df523ab96295038c914"
server_key_version: "encryptionkey@0"
I20260811 19:00:53.457270 23713 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 19:00:53.459926 23728 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:53.461241 23713 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 19:00:53.461380 23713 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474792496492-13748-0/minicluster-data/master-0/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474792496492-13748-0/minicluster-data/master-0/wal
uuid: "1fb2c37c64424c7fb0e7a17f5f454c43"
format_stamp: "Formatted at 2026-08-11 19:00:53 on dist-test-slave-0kls"
server_key: "d738208d3f6a0d094531fe5b6aed4a7c"
server_key_iv: "0a6fdc966e7d5df523ab96295038c914"
server_key_version: "encryptionkey@0"
I20260811 19:00:53.461501 23713 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474792496492-13748-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474792496492-13748-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474792496492-13748-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:00:53.493403 23713 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:00:53.494232 23713 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:00:53.494457 23713 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:00:53.502002 23713 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.62:36033
I20260811 19:00:53.502020 23786 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.62:36033 every 8 connection(s)
I20260811 19:00:53.503008 23713 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
I20260811 19:00:53.506390 23787 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:00:53.510470 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 23713
I20260811 19:00:53.510609 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474792496492-13748-0/minicluster-data/master-0/wal/instance
I20260811 19:00:53.510937 13748 external_mini_cluster.cc:1428] Setting key fd120aa7154027236f1bd47140c76056
I20260811 19:00:53.512761 23787 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1fb2c37c64424c7fb0e7a17f5f454c43: Bootstrap starting.
I20260811 19:00:53.515093 23787 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1fb2c37c64424c7fb0e7a17f5f454c43: Neither blocks nor log segments found. Creating new log.
I20260811 19:00:53.515877 23787 log.cc:824] T 00000000000000000000000000000000 P 1fb2c37c64424c7fb0e7a17f5f454c43: Log is configured to *not* fsync() on all Append() calls
I20260811 19:00:53.518139 23787 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1fb2c37c64424c7fb0e7a17f5f454c43: No bootstrap required, opened a new log
I20260811 19:00:53.520968 23790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:53.512491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47290 (local address 127.13.109.62:36033)
0811 19:00:53.513201 (+ 710us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:53.513218 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:53.513256 (+ 38us) server_negotiation.cc:407] Connection header received
0811 19:00:53.513951 (+ 695us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:53.513969 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:53.514373 (+ 404us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:53.514724 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:53.515200 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:53.516292 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:53.517491 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:53.518008 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:53.518137 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:53.519024 (+ 887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:53.519029 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:53.519042 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:00:53.519434 (+ 392us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:53.519489 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:00:53.519508 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:53.519587 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:53.519768 (+ 181us) server_negotiation.cc:299] Negotiation successful
0811 19:00:53.520060 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":160,"threads_started":1}
I20260811 19:00:53.521600 23787 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1fb2c37c64424c7fb0e7a17f5f454c43 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fb2c37c64424c7fb0e7a17f5f454c43" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 36033 } }
I20260811 19:00:53.521873 23787 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1fb2c37c64424c7fb0e7a17f5f454c43 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:00:53.521955 23787 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1fb2c37c64424c7fb0e7a17f5f454c43 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fb2c37c64424c7fb0e7a17f5f454c43, State: Initialized, Role: FOLLOWER
I20260811 19:00:53.522372 23787 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1fb2c37c64424c7fb0e7a17f5f454c43 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fb2c37c64424c7fb0e7a17f5f454c43" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 36033 } }
I20260811 19:00:53.522527 23787 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1fb2c37c64424c7fb0e7a17f5f454c43 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:00:53.522617 23787 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1fb2c37c64424c7fb0e7a17f5f454c43 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:00:53.522738 23787 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1fb2c37c64424c7fb0e7a17f5f454c43 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:00:53.523687 23787 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1fb2c37c64424c7fb0e7a17f5f454c43 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fb2c37c64424c7fb0e7a17f5f454c43" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 36033 } }
I20260811 19:00:53.524027 23787 leader_election.cc:302] T 00000000000000000000000000000000 P 1fb2c37c64424c7fb0e7a17f5f454c43 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1fb2c37c64424c7fb0e7a17f5f454c43; no voters:
I20260811 19:00:53.524341 23787 leader_election.cc:288] T 00000000000000000000000000000000 P 1fb2c37c64424c7fb0e7a17f5f454c43 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:00:53.524456 23792 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1fb2c37c64424c7fb0e7a17f5f454c43 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:00:53.524725 23792 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1fb2c37c64424c7fb0e7a17f5f454c43 [term 1 LEADER]: Becoming Leader. State: Replica: 1fb2c37c64424c7fb0e7a17f5f454c43, State: Running, Role: LEADER
I20260811 19:00:53.525031 23792 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1fb2c37c64424c7fb0e7a17f5f454c43 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fb2c37c64424c7fb0e7a17f5f454c43" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 36033 } }
I20260811 19:00:53.525611 23787 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1fb2c37c64424c7fb0e7a17f5f454c43 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:00:53.526840 23795 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1fb2c37c64424c7fb0e7a17f5f454c43 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1fb2c37c64424c7fb0e7a17f5f454c43" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fb2c37c64424c7fb0e7a17f5f454c43" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 36033 } } }
I20260811 19:00:53.526990 23795 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1fb2c37c64424c7fb0e7a17f5f454c43 [sys.catalog]: This master's current role is: LEADER
I20260811 19:00:53.527275 23796 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1fb2c37c64424c7fb0e7a17f5f454c43 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1fb2c37c64424c7fb0e7a17f5f454c43. Latest consensus state: current_term: 1 leader_uuid: "1fb2c37c64424c7fb0e7a17f5f454c43" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fb2c37c64424c7fb0e7a17f5f454c43" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 36033 } } }
I20260811 19:00:53.527370 23796 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1fb2c37c64424c7fb0e7a17f5f454c43 [sys.catalog]: This master's current role is: LEADER
I20260811 19:00:53.527496 23803 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:00:53.530718 23803 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:00:53.539381 23803 catalog_manager.cc:1347] Generated new cluster ID: 709481af08dc4e99a1854076439d6f24
I20260811 19:00:53.539474 23803 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:00:53.571085 23803 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:00:53.572117 23803 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:00:53.578814 23803 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1fb2c37c64424c7fb0e7a17f5f454c43: Generated new TSK 0
I20260811 19:00:53.579761 23803 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 19:00:53.589967 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.1:0
--local_ip_for_outbound_sockets=127.13.109.1
--webserver_interface=127.13.109.1
--webserver_port=0
--tserver_master_addrs=127.13.109.62:36033
--builtin_ntp_servers=127.13.109.20:45441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260811 19:00:53.727756 23814 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:00:53.728060 23814 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:00:53.728102 23814 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:00:53.732285 23814 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:00:53.732354 23814 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:00:53.732412 23814 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.1
I20260811 19:00:53.737385 23814 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:45441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.109.1
--webserver_port=0
--tserver_master_addrs=127.13.109.62:36033
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23814
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.1
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:00:53.738667 23814 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:00:53.740129 23814 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:00:53.750424 23820 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:53.750483 23814 server_base.cc:1034] running on GCE node
W20260811 19:00:53.750439 23823 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:53.750429 23821 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:53.751096 23814 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:00:53.751813 23814 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:00:53.753000 23814 hybrid_clock.cc:648] HybridClock initialized: now 1786474853752981 us; error 32 us; skew 500 ppm
I20260811 19:00:53.755090 23814 webserver.cc:460] Webserver started at http://127.13.109.1:39957/ using document root <none> and password file <none>
I20260811 19:00:53.755738 23814 fs_manager.cc:359] Metadata directory not provided
I20260811 19:00:53.755837 23814 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:00:53.756088 23814 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:00:53.758400 23814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-0/data/instance:
uuid: "b3c17627a5f84fcca68e1ede4ba99d39"
format_stamp: "Formatted at 2026-08-11 19:00:53 on dist-test-slave-0kls"
server_key: "74ee69d2fa9c7f8b5a6722d856ec0191"
server_key_iv: "9cbd920620c6c73c4572c94b074136fb"
server_key_version: "encryptionkey@0"
I20260811 19:00:53.759013 23814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-0/wal/instance:
uuid: "b3c17627a5f84fcca68e1ede4ba99d39"
format_stamp: "Formatted at 2026-08-11 19:00:53 on dist-test-slave-0kls"
server_key: "74ee69d2fa9c7f8b5a6722d856ec0191"
server_key_iv: "9cbd920620c6c73c4572c94b074136fb"
server_key_version: "encryptionkey@0"
I20260811 19:00:53.763113 23814 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 19:00:53.766105 23830 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:53.767444 23814 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 19:00:53.767627 23814 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-0/wal
uuid: "b3c17627a5f84fcca68e1ede4ba99d39"
format_stamp: "Formatted at 2026-08-11 19:00:53 on dist-test-slave-0kls"
server_key: "74ee69d2fa9c7f8b5a6722d856ec0191"
server_key_iv: "9cbd920620c6c73c4572c94b074136fb"
server_key_version: "encryptionkey@0"
I20260811 19:00:53.767779 23814 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:00:53.789250 23814 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:00:53.790046 23814 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:00:53.790213 23814 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:00:53.790812 23814 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:00:53.791796 23814 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:00:53.791846 23814 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:53.791889 23814 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:00:53.791910 23814 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:53.801776 23814 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.1:33029
I20260811 19:00:53.801813 23955 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.1:33029 every 8 connection(s)
I20260811 19:00:53.802793 23814 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
I20260811 19:00:53.808879 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 23814
I20260811 19:00:53.809005 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-0/wal/instance
I20260811 19:00:53.809357 13748 external_mini_cluster.cc:1428] Setting key 5ec443f8d0b655a1704d08f27cc62bbb
I20260811 19:00:53.811539 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.2:0
--local_ip_for_outbound_sockets=127.13.109.2
--webserver_interface=127.13.109.2
--webserver_port=0
--tserver_master_addrs=127.13.109.62:36033
--builtin_ntp_servers=127.13.109.20:45441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260811 19:00:53.813306 23790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:53.804784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.1:57759 (local address 127.13.109.62:36033)
0811 19:00:53.804938 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:53.804942 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:53.806172 (+ 1230us) server_negotiation.cc:407] Connection header received
0811 19:00:53.807581 (+ 1409us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:53.807584 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:53.807637 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:53.807721 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:53.808661 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:53.809478 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:53.811078 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:53.811451 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:53.811572 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:53.812737 (+ 1165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:53.812744 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:53.812750 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:00:53.812768 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:53.812794 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:00:53.812807 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:53.812884 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:53.813032 (+ 148us) server_negotiation.cc:299] Negotiation successful
0811 19:00:53.813112 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 19:00:53.814488 23958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:53.805242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:36033 (local address 127.13.109.1:57759)
0811 19:00:53.806016 (+ 774us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:53.806059 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:53.807385 (+ 1326us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:53.807885 (+ 500us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:53.807891 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:53.807973 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:53.808490 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:53.808503 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:53.809611 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:53.809614 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:53.810961 (+ 1347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:53.810971 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:53.811598 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:53.811601 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:53.811835 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:53.812465 (+ 630us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 19:00:53.812467 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:53.812509 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 19:00:53.812574 (+ 65us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:00:53.812898 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:00:53.812906 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:00:53.812910 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 19:00:53.813001 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 19:00:53.813361 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":121,"threads_started":1}
I20260811 19:00:53.815790 23956 heartbeater.cc:347] Connected to a master server at 127.13.109.62:36033
I20260811 19:00:53.816033 23956 heartbeater.cc:464] Registering TS with master...
I20260811 19:00:53.816603 23956 heartbeater.cc:511] Master 127.13.109.62:36033 requested a full tablet report, sending...
I20260811 19:00:53.818388 23746 ts_manager.cc:194] Registered new tserver with Master: b3c17627a5f84fcca68e1ede4ba99d39 (127.13.109.1:33029)
I20260811 19:00:53.820200 23746 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.109.1:57759
W20260811 19:00:53.934196 23959 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:00:53.934508 23959 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:00:53.934548 23959 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:00:53.938768 23959 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:00:53.938874 23959 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:00:53.938969 23959 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.2
I20260811 19:00:53.944118 23959 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:45441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.109.2
--webserver_port=0
--tserver_master_addrs=127.13.109.62:36033
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23959
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.2
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:00:53.945462 23959 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:00:53.947120 23959 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 19:00:53.958215 23959 server_base.cc:1034] running on GCE node
W20260811 19:00:53.958168 23968 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:53.958168 23965 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:53.958168 23964 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:53.958917 23959 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:00:53.959708 23959 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:00:53.960924 23959 hybrid_clock.cc:648] HybridClock initialized: now 1786474853960904 us; error 37 us; skew 500 ppm
I20260811 19:00:53.963107 23959 webserver.cc:460] Webserver started at http://127.13.109.2:32963/ using document root <none> and password file <none>
I20260811 19:00:53.963807 23959 fs_manager.cc:359] Metadata directory not provided
I20260811 19:00:53.963920 23959 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:00:53.964191 23959 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:00:53.966717 23959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-1/data/instance:
uuid: "1d62b52f7c7749d8afa3eb8366c627ba"
format_stamp: "Formatted at 2026-08-11 19:00:53 on dist-test-slave-0kls"
server_key: "065332878345963be24526813a27d93d"
server_key_iv: "17929e3309e9e12a4dd6b7ae1167c496"
server_key_version: "encryptionkey@0"
I20260811 19:00:53.967386 23959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-1/wal/instance:
uuid: "1d62b52f7c7749d8afa3eb8366c627ba"
format_stamp: "Formatted at 2026-08-11 19:00:53 on dist-test-slave-0kls"
server_key: "065332878345963be24526813a27d93d"
server_key_iv: "17929e3309e9e12a4dd6b7ae1167c496"
server_key_version: "encryptionkey@0"
I20260811 19:00:53.971714 23959 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260811 19:00:53.974872 23975 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:53.976333 23959 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 19:00:53.976502 23959 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-1/wal
uuid: "1d62b52f7c7749d8afa3eb8366c627ba"
format_stamp: "Formatted at 2026-08-11 19:00:53 on dist-test-slave-0kls"
server_key: "065332878345963be24526813a27d93d"
server_key_iv: "17929e3309e9e12a4dd6b7ae1167c496"
server_key_version: "encryptionkey@0"
I20260811 19:00:53.976650 23959 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:00:53.997154 23959 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:00:53.997992 23959 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:00:53.998219 23959 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:00:53.998890 23959 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:00:53.999984 23959 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:00:54.000059 23959 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:54.000161 23959 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:00:54.000209 23959 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:54.011720 23959 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.2:41211
I20260811 19:00:54.011754 24098 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.2:41211 every 8 connection(s)
I20260811 19:00:54.012698 23959 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
I20260811 19:00:54.020588 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 23959
I20260811 19:00:54.020736 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-1/wal/instance
I20260811 19:00:54.021024 13748 external_mini_cluster.cc:1428] Setting key 2c7918ada96fbc11c86f0cab100df317
I20260811 19:00:54.022716 23790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:54.014679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.2:55971 (local address 127.13.109.62:36033)
0811 19:00:54.014826 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:54.014830 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:54.015972 (+ 1142us) server_negotiation.cc:407] Connection header received
0811 19:00:54.017341 (+ 1369us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:54.017343 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:54.017391 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:54.017470 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:54.018459 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:54.018988 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:54.020232 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:54.020592 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:54.020718 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:54.022104 (+ 1386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:54.022108 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:54.022111 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:00:54.022130 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:54.022150 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:00:54.022163 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:54.022246 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:54.022475 (+ 229us) server_negotiation.cc:299] Negotiation successful
0811 19:00:54.022545 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 19:00:54.023322 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.3:0
--local_ip_for_outbound_sockets=127.13.109.3
--webserver_interface=127.13.109.3
--webserver_port=0
--tserver_master_addrs=127.13.109.62:36033
--builtin_ntp_servers=127.13.109.20:45441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260811 19:00:54.023939 24101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:54.015101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:36033 (local address 127.13.109.2:55971)
0811 19:00:54.015806 (+ 705us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:54.015859 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:54.017149 (+ 1290us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:54.017650 (+ 501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:54.017657 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:54.017741 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:54.018327 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:54.018342 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:54.019107 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:54.019110 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:54.020088 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:54.020102 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:54.020832 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:54.020839 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:54.021015 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:54.021806 (+ 791us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 19:00:54.021808 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:54.021867 (+ 59us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 19:00:54.021945 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:00:54.022262 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:00:54.022272 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:00:54.022284 (+ 12us) client_negotiation.cc:769] Sending connection context
0811 19:00:54.022433 (+ 149us) client_negotiation.cc:240] Negotiation successful
0811 19:00:54.022785 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":125,"threads_started":1}
I20260811 19:00:54.025187 24099 heartbeater.cc:347] Connected to a master server at 127.13.109.62:36033
I20260811 19:00:54.025486 24099 heartbeater.cc:464] Registering TS with master...
I20260811 19:00:54.026245 24099 heartbeater.cc:511] Master 127.13.109.62:36033 requested a full tablet report, sending...
I20260811 19:00:54.027599 23746 ts_manager.cc:194] Registered new tserver with Master: 1d62b52f7c7749d8afa3eb8366c627ba (127.13.109.2:41211)
I20260811 19:00:54.028271 23746 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.109.2:55971
W20260811 19:00:54.144531 24102 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:00:54.144881 24102 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:00:54.144953 24102 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:00:54.149231 24102 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:00:54.149338 24102 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:00:54.149437 24102 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.3
I20260811 19:00:54.154690 24102 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:45441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.109.3
--webserver_port=0
--tserver_master_addrs=127.13.109.62:36033
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24102
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.3
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:00:54.155874 24102 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:00:54.157318 24102 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:00:54.167704 24107 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:54.167724 24108 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:54.167778 24102 server_base.cc:1034] running on GCE node
W20260811 19:00:54.167675 24111 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:54.168365 24102 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:00:54.169071 24102 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:00:54.170259 24102 hybrid_clock.cc:648] HybridClock initialized: now 1786474854170240 us; error 31 us; skew 500 ppm
I20260811 19:00:54.172302 24102 webserver.cc:460] Webserver started at http://127.13.109.3:37879/ using document root <none> and password file <none>
I20260811 19:00:54.172966 24102 fs_manager.cc:359] Metadata directory not provided
I20260811 19:00:54.173065 24102 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:00:54.173309 24102 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:00:54.175573 24102 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-2/data/instance:
uuid: "d960810113534210b970ee4e87b2b422"
format_stamp: "Formatted at 2026-08-11 19:00:54 on dist-test-slave-0kls"
server_key: "6a7f047f1d719dd5bdd90ce68112e196"
server_key_iv: "f30b0b928b3ba07e562a2af91e1ab397"
server_key_version: "encryptionkey@0"
I20260811 19:00:54.176188 24102 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-2/wal/instance:
uuid: "d960810113534210b970ee4e87b2b422"
format_stamp: "Formatted at 2026-08-11 19:00:54 on dist-test-slave-0kls"
server_key: "6a7f047f1d719dd5bdd90ce68112e196"
server_key_iv: "f30b0b928b3ba07e562a2af91e1ab397"
server_key_version: "encryptionkey@0"
I20260811 19:00:54.180107 24102 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260811 19:00:54.182984 24118 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:54.184274 24102 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 19:00:54.184407 24102 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-2/wal
uuid: "d960810113534210b970ee4e87b2b422"
format_stamp: "Formatted at 2026-08-11 19:00:54 on dist-test-slave-0kls"
server_key: "6a7f047f1d719dd5bdd90ce68112e196"
server_key_iv: "f30b0b928b3ba07e562a2af91e1ab397"
server_key_version: "encryptionkey@0"
I20260811 19:00:54.184527 24102 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:00:54.212733 24102 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:00:54.213472 24102 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:00:54.213644 24102 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:00:54.214303 24102 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:00:54.215266 24102 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:00:54.215315 24102 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:54.215358 24102 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:00:54.215377 24102 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:54.224009 24102 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.3:38405
I20260811 19:00:54.224036 24248 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.3:38405 every 8 connection(s)
I20260811 19:00:54.224951 24102 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
I20260811 19:00:54.232304 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 24102
I20260811 19:00:54.232445 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-2/wal/instance
I20260811 19:00:54.232774 13748 external_mini_cluster.cc:1428] Setting key 40552e55375bb7ff97f326ccab38cbbc
I20260811 19:00:54.235255 23790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:54.227210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.3:40541 (local address 127.13.109.62:36033)
0811 19:00:54.227401 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:54.227407 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:54.228528 (+ 1121us) server_negotiation.cc:407] Connection header received
0811 19:00:54.229889 (+ 1361us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:54.229892 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:54.229946 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:54.230023 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:54.230908 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:54.231438 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:54.232984 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:54.233275 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:54.233394 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:54.234739 (+ 1345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:54.234743 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:54.234745 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:00:54.234759 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:54.234785 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:00:54.234794 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:54.234866 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:54.235016 (+ 150us) server_negotiation.cc:299] Negotiation successful
0811 19:00:54.235102 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260811 19:00:54.236212 24252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:54.227735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:36033 (local address 127.13.109.3:40541)
0811 19:00:54.228386 (+ 651us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:54.228427 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:54.229672 (+ 1245us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:54.230188 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:54.230195 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:54.230277 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:54.230783 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:54.230796 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:54.231545 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:54.231548 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:54.232796 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:54.232811 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:54.233467 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:54.233472 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:54.233667 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:54.234452 (+ 785us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 19:00:54.234454 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:54.234512 (+ 58us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 19:00:54.234586 (+ 74us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:00:54.234883 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:00:54.234890 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:00:54.234894 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 19:00:54.235009 (+ 115us) client_negotiation.cc:240] Negotiation successful
0811 19:00:54.235357 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":115,"threads_started":1}
I20260811 19:00:54.237366 24249 heartbeater.cc:347] Connected to a master server at 127.13.109.62:36033
I20260811 19:00:54.237627 24249 heartbeater.cc:464] Registering TS with master...
I20260811 19:00:54.238248 24249 heartbeater.cc:511] Master 127.13.109.62:36033 requested a full tablet report, sending...
I20260811 19:00:54.239293 23746 ts_manager.cc:194] Registered new tserver with Master: d960810113534210b970ee4e87b2b422 (127.13.109.3:38405)
I20260811 19:00:54.239888 23746 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.109.3:40541
I20260811 19:00:54.247037 13748 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 19:00:54.250026 23790 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:00:54.248962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47306 (local address 127.13.109.62:36033)
0811 19:00:54.249091 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:54.249097 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:54.249306 (+ 209us) server_negotiation.cc:407] Connection header received
0811 19:00:54.249449 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:54.249451 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:54.249494 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:54.249574 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:54.249882 (+ 308us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47306: BlockingRecv error: recv got EOF from 127.0.0.1:47306 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
I20260811 19:00:54.250147 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 23814
I20260811 19:00:54.261019 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 23959
I20260811 19:00:54.269470 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 24102
I20260811 19:00:54.278496 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 23713
2026-08-11T19:00:54Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (1009 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-11T19:00:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:00:54Z Disabled control of system clock
I20260811 19:00:54.312924 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474792496492-13748-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474792496492-13748-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474792496492-13748-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.62:46361
--webserver_interface=127.13.109.62
--webserver_port=0
--builtin_ntp_servers=127.13.109.20:38177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.109.62:46361
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 19:00:54.429459 24261 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:00:54.429769 24261 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:00:54.429905 24261 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:00:54.434068 24261 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:00:54.434146 24261 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:00:54.434163 24261 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:00:54.434180 24261 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:00:54.434196 24261 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:00:54.439270 24261 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:38177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474792496492-13748-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474792496492-13748-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.109.62:46361
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.62:46361
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.109.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24261
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474792496492-13748-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:00:54.440485 24261 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:00:54.441927 24261 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:00:54.450894 24268 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:54.450881 24267 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:54.450875 24270 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:54.451272 24261 server_base.cc:1034] running on GCE node
I20260811 19:00:54.451925 24261 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:00:54.453096 24261 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:00:54.454289 24261 hybrid_clock.cc:648] HybridClock initialized: now 1786474854454279 us; error 32 us; skew 500 ppm
I20260811 19:00:54.456346 24261 webserver.cc:460] Webserver started at http://127.13.109.62:42263/ using document root <none> and password file <none>
I20260811 19:00:54.456969 24261 fs_manager.cc:359] Metadata directory not provided
I20260811 19:00:54.457068 24261 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:00:54.457317 24261 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:00:54.459578 24261 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474792496492-13748-0/minicluster-data/master-0/data/instance:
uuid: "4cd36b1fa6f8464c9c3fd767d91fc424"
format_stamp: "Formatted at 2026-08-11 19:00:54 on dist-test-slave-0kls"
server_key: "29ca8b6dbe9559b034a0e4e517590ac3"
server_key_iv: "7aa3fcd63aacd724b4f4265514b0a8a9"
server_key_version: "encryptionkey@0"
I20260811 19:00:54.460290 24261 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474792496492-13748-0/minicluster-data/master-0/wal/instance:
uuid: "4cd36b1fa6f8464c9c3fd767d91fc424"
format_stamp: "Formatted at 2026-08-11 19:00:54 on dist-test-slave-0kls"
server_key: "29ca8b6dbe9559b034a0e4e517590ac3"
server_key_iv: "7aa3fcd63aacd724b4f4265514b0a8a9"
server_key_version: "encryptionkey@0"
I20260811 19:00:54.464327 24261 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 19:00:54.466938 24279 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:54.468261 24261 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 19:00:54.468410 24261 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474792496492-13748-0/minicluster-data/master-0/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474792496492-13748-0/minicluster-data/master-0/wal
uuid: "4cd36b1fa6f8464c9c3fd767d91fc424"
format_stamp: "Formatted at 2026-08-11 19:00:54 on dist-test-slave-0kls"
server_key: "29ca8b6dbe9559b034a0e4e517590ac3"
server_key_iv: "7aa3fcd63aacd724b4f4265514b0a8a9"
server_key_version: "encryptionkey@0"
I20260811 19:00:54.468537 24261 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474792496492-13748-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474792496492-13748-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474792496492-13748-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:00:54.473678 24261 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:00:54.474349 24261 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:00:54.474536 24261 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:00:54.482180 24261 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.62:46361
I20260811 19:00:54.482235 24340 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.62:46361 every 8 connection(s)
I20260811 19:00:54.483294 24261 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
I20260811 19:00:54.486940 24342 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:00:54.489724 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 24261
I20260811 19:00:54.489884 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474792496492-13748-0/minicluster-data/master-0/wal/instance
I20260811 19:00:54.490172 13748 external_mini_cluster.cc:1428] Setting key 03e0a14794bf739a1e8acecf3d7320e9
I20260811 19:00:54.493335 24342 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4cd36b1fa6f8464c9c3fd767d91fc424: Bootstrap starting.
I20260811 19:00:54.495951 24342 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4cd36b1fa6f8464c9c3fd767d91fc424: Neither blocks nor log segments found. Creating new log.
I20260811 19:00:54.496001 24345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:54.491766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45444 (local address 127.13.109.62:46361)
0811 19:00:54.492475 (+ 709us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:54.492490 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:54.492528 (+ 38us) server_negotiation.cc:407] Connection header received
0811 19:00:54.493158 (+ 630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:54.493185 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:54.493597 (+ 412us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:54.494018 (+ 421us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:54.494749 (+ 731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:54.494754 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:54.494766 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:00:54.494775 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:54.494827 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:00:54.494843 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:54.494993 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:54.495066 (+ 73us) server_negotiation.cc:299] Negotiation successful
0811 19:00:54.495310 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":149,"threads_started":1}
I20260811 19:00:54.497040 24342 log.cc:824] T 00000000000000000000000000000000 P 4cd36b1fa6f8464c9c3fd767d91fc424: Log is configured to *not* fsync() on all Append() calls
I20260811 19:00:54.499230 24342 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4cd36b1fa6f8464c9c3fd767d91fc424: No bootstrap required, opened a new log
I20260811 19:00:54.503149 24342 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4cd36b1fa6f8464c9c3fd767d91fc424 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cd36b1fa6f8464c9c3fd767d91fc424" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 46361 } }
I20260811 19:00:54.503404 24342 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4cd36b1fa6f8464c9c3fd767d91fc424 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:00:54.503448 24342 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4cd36b1fa6f8464c9c3fd767d91fc424 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cd36b1fa6f8464c9c3fd767d91fc424, State: Initialized, Role: FOLLOWER
I20260811 19:00:54.503885 24342 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4cd36b1fa6f8464c9c3fd767d91fc424 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cd36b1fa6f8464c9c3fd767d91fc424" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 46361 } }
I20260811 19:00:54.504015 24342 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4cd36b1fa6f8464c9c3fd767d91fc424 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:00:54.504065 24342 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4cd36b1fa6f8464c9c3fd767d91fc424 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:00:54.504140 24342 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4cd36b1fa6f8464c9c3fd767d91fc424 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:00:54.505111 24342 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4cd36b1fa6f8464c9c3fd767d91fc424 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cd36b1fa6f8464c9c3fd767d91fc424" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 46361 } }
I20260811 19:00:54.505429 24342 leader_election.cc:302] T 00000000000000000000000000000000 P 4cd36b1fa6f8464c9c3fd767d91fc424 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4cd36b1fa6f8464c9c3fd767d91fc424; no voters:
I20260811 19:00:54.505722 24342 leader_election.cc:288] T 00000000000000000000000000000000 P 4cd36b1fa6f8464c9c3fd767d91fc424 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:00:54.505875 24347 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4cd36b1fa6f8464c9c3fd767d91fc424 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:00:54.506150 24347 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4cd36b1fa6f8464c9c3fd767d91fc424 [term 1 LEADER]: Becoming Leader. State: Replica: 4cd36b1fa6f8464c9c3fd767d91fc424, State: Running, Role: LEADER
I20260811 19:00:54.506498 24347 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4cd36b1fa6f8464c9c3fd767d91fc424 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cd36b1fa6f8464c9c3fd767d91fc424" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 46361 } }
I20260811 19:00:54.507217 24342 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4cd36b1fa6f8464c9c3fd767d91fc424 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:00:54.508303 24348 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4cd36b1fa6f8464c9c3fd767d91fc424 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4cd36b1fa6f8464c9c3fd767d91fc424" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cd36b1fa6f8464c9c3fd767d91fc424" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 46361 } } }
I20260811 19:00:54.508460 24348 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4cd36b1fa6f8464c9c3fd767d91fc424 [sys.catalog]: This master's current role is: LEADER
I20260811 19:00:54.508721 24349 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4cd36b1fa6f8464c9c3fd767d91fc424 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4cd36b1fa6f8464c9c3fd767d91fc424. Latest consensus state: current_term: 1 leader_uuid: "4cd36b1fa6f8464c9c3fd767d91fc424" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cd36b1fa6f8464c9c3fd767d91fc424" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 46361 } } }
I20260811 19:00:54.508827 24356 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:00:54.508831 24349 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4cd36b1fa6f8464c9c3fd767d91fc424 [sys.catalog]: This master's current role is: LEADER
I20260811 19:00:54.512293 24356 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:00:54.518293 24356 catalog_manager.cc:1347] Generated new cluster ID: 0d98b1af235247e5b24fe28611f6f336
I20260811 19:00:54.518388 24356 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:00:54.532928 24356 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:00:54.533281 24356 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:00:54.538873 24356 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4cd36b1fa6f8464c9c3fd767d91fc424: Generated new TSK 0
I20260811 19:00:54.539637 24356 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 19:00:54.547982 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.1:0
--local_ip_for_outbound_sockets=127.13.109.1
--webserver_interface=127.13.109.1
--webserver_port=0
--tserver_master_addrs=127.13.109.62:46361
--builtin_ntp_servers=127.13.109.20:38177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 19:00:54.669286 24367 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:00:54.669579 24367 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:00:54.669621 24367 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:00:54.673967 24367 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:00:54.674058 24367 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:00:54.674165 24367 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.1
I20260811 19:00:54.679210 24367 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:38177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.109.1
--webserver_port=0
--tserver_master_addrs=127.13.109.62:46361
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24367
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.1
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:00:54.680460 24367 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:00:54.681981 24367 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:00:54.692119 24376 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:54.692111 24375 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:54.692258 24367 server_base.cc:1034] running on GCE node
W20260811 19:00:54.692121 24378 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:54.692822 24367 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:00:54.693492 24367 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:00:54.694682 24367 hybrid_clock.cc:648] HybridClock initialized: now 1786474854694642 us; error 52 us; skew 500 ppm
I20260811 19:00:54.696597 24367 webserver.cc:460] Webserver started at http://127.13.109.1:44949/ using document root <none> and password file <none>
I20260811 19:00:54.697247 24367 fs_manager.cc:359] Metadata directory not provided
I20260811 19:00:54.697328 24367 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:00:54.697576 24367 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:00:54.699873 24367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-0/data/instance:
uuid: "28cab8fbdce64a7d9cf9b739a8f59635"
format_stamp: "Formatted at 2026-08-11 19:00:54 on dist-test-slave-0kls"
server_key: "fd95afe35074e4c9e66e6994835b994f"
server_key_iv: "722d370ff721257a7ab848611f58d5c2"
server_key_version: "encryptionkey@0"
I20260811 19:00:54.700477 24367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-0/wal/instance:
uuid: "28cab8fbdce64a7d9cf9b739a8f59635"
format_stamp: "Formatted at 2026-08-11 19:00:54 on dist-test-slave-0kls"
server_key: "fd95afe35074e4c9e66e6994835b994f"
server_key_iv: "722d370ff721257a7ab848611f58d5c2"
server_key_version: "encryptionkey@0"
I20260811 19:00:54.704525 24367 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 19:00:54.707521 24384 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:54.708948 24367 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 19:00:54.709101 24367 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-0/wal
uuid: "28cab8fbdce64a7d9cf9b739a8f59635"
format_stamp: "Formatted at 2026-08-11 19:00:54 on dist-test-slave-0kls"
server_key: "fd95afe35074e4c9e66e6994835b994f"
server_key_iv: "722d370ff721257a7ab848611f58d5c2"
server_key_version: "encryptionkey@0"
I20260811 19:00:54.709236 24367 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:00:54.714309 24367 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:00:54.714974 24367 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:00:54.715175 24367 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:00:54.715770 24367 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:00:54.716784 24367 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:00:54.716856 24367 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:54.716928 24367 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:00:54.716969 24367 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:54.726186 24367 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.1:46035
I20260811 19:00:54.726195 24513 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.1:46035 every 8 connection(s)
I20260811 19:00:54.727284 24367 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
I20260811 19:00:54.734025 24345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:54.729351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.1:58849 (local address 127.13.109.62:46361)
0811 19:00:54.729542 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:54.729548 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:54.730823 (+ 1275us) server_negotiation.cc:407] Connection header received
0811 19:00:54.732179 (+ 1356us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:54.732182 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:54.732225 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:54.732308 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:54.733537 (+ 1229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:54.733540 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:54.733542 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:00:54.733552 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:54.733568 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:00:54.733575 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:54.733667 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:54.733848 (+ 181us) server_negotiation.cc:299] Negotiation successful
0811 19:00:54.733902 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 19:00:54.735003 24517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:54.729887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:46361 (local address 127.13.109.1:58849)
0811 19:00:54.730641 (+ 754us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:54.730694 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:54.731955 (+ 1261us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:54.732499 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:54.732507 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:54.732606 (+ 99us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:54.733264 (+ 658us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 19:00:54.733265 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:54.733312 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 19:00:54.733391 (+ 79us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:00:54.733660 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:00:54.733667 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:00:54.733672 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 19:00:54.733880 (+ 208us) client_negotiation.cc:240] Negotiation successful
0811 19:00:54.734131 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":130,"threads_started":1}
I20260811 19:00:54.735795 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 24367
I20260811 19:00:54.735924 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-0/wal/instance
I20260811 19:00:54.736291 13748 external_mini_cluster.cc:1428] Setting key d7bf85c97a5ecee3cc4443bea971b365
I20260811 19:00:54.736496 24514 heartbeater.cc:347] Connected to a master server at 127.13.109.62:46361
I20260811 19:00:54.736773 24514 heartbeater.cc:464] Registering TS with master...
I20260811 19:00:54.737285 24514 heartbeater.cc:511] Master 127.13.109.62:46361 requested a full tablet report, sending...
I20260811 19:00:54.738509 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.2:0
--local_ip_for_outbound_sockets=127.13.109.2
--webserver_interface=127.13.109.2
--webserver_port=0
--tserver_master_addrs=127.13.109.62:46361
--builtin_ntp_servers=127.13.109.20:38177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260811 19:00:54.739049 24298 ts_manager.cc:194] Registered new tserver with Master: 28cab8fbdce64a7d9cf9b739a8f59635 (127.13.109.1:46035)
W20260811 19:00:54.859452 24518 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:00:54.859760 24518 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:00:54.859830 24518 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:00:54.863991 24518 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:00:54.864048 24518 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:00:54.864137 24518 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.2
I20260811 19:00:54.869292 24518 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:38177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.109.2
--webserver_port=0
--tserver_master_addrs=127.13.109.62:46361
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24518
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.2
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:00:54.870529 24518 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:00:54.871963 24518 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:00:54.882193 24523 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:54.882428 24518 server_base.cc:1034] running on GCE node
W20260811 19:00:54.882193 24524 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:54.882222 24527 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:54.883015 24518 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:00:54.883733 24518 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:00:54.884922 24518 hybrid_clock.cc:648] HybridClock initialized: now 1786474854884906 us; error 32 us; skew 500 ppm
I20260811 19:00:54.891371 24518 webserver.cc:460] Webserver started at http://127.13.109.2:37217/ using document root <none> and password file <none>
I20260811 19:00:54.892318 24518 fs_manager.cc:359] Metadata directory not provided
I20260811 19:00:54.892416 24518 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:00:54.892721 24518 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:00:54.896204 24518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-1/data/instance:
uuid: "8559057a68df4fee8648cb7eb058e674"
format_stamp: "Formatted at 2026-08-11 19:00:54 on dist-test-slave-0kls"
server_key: "991441b8b85d114cb1c12a76a081d1bf"
server_key_iv: "1e391937f759e4e68bfb9770c4cb5ede"
server_key_version: "encryptionkey@0"
I20260811 19:00:54.897078 24518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-1/wal/instance:
uuid: "8559057a68df4fee8648cb7eb058e674"
format_stamp: "Formatted at 2026-08-11 19:00:54 on dist-test-slave-0kls"
server_key: "991441b8b85d114cb1c12a76a081d1bf"
server_key_iv: "1e391937f759e4e68bfb9770c4cb5ede"
server_key_version: "encryptionkey@0"
I20260811 19:00:54.902928 24518 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.003s
I20260811 19:00:54.907439 24534 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:54.909433 24518 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.004s
I20260811 19:00:54.909605 24518 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-1/wal
uuid: "8559057a68df4fee8648cb7eb058e674"
format_stamp: "Formatted at 2026-08-11 19:00:54 on dist-test-slave-0kls"
server_key: "991441b8b85d114cb1c12a76a081d1bf"
server_key_iv: "1e391937f759e4e68bfb9770c4cb5ede"
server_key_version: "encryptionkey@0"
I20260811 19:00:54.909822 24518 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:00:54.917109 24518 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:00:54.918072 24518 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:00:54.918318 24518 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:00:54.919094 24518 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:00:54.920481 24518 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:00:54.920543 24518 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:54.920601 24518 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:00:54.920634 24518 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:54.934377 24518 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.2:45731
I20260811 19:00:54.934389 24658 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.2:45731 every 8 connection(s)
I20260811 19:00:54.935915 24518 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
I20260811 19:00:54.937136 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 24518
I20260811 19:00:54.937304 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-1/wal/instance
I20260811 19:00:54.937719 13748 external_mini_cluster.cc:1428] Setting key b33e6b9292773b669beb005c8aabfb95
I20260811 19:00:54.940464 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.3:0
--local_ip_for_outbound_sockets=127.13.109.3
--webserver_interface=127.13.109.3
--webserver_port=0
--tserver_master_addrs=127.13.109.62:46361
--builtin_ntp_servers=127.13.109.20:38177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260811 19:00:54.950673 24345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:54.940017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.2:60905 (local address 127.13.109.62:46361)
0811 19:00:54.940142 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:54.940148 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:54.943357 (+ 3209us) server_negotiation.cc:407] Connection header received
0811 19:00:54.945103 (+ 1746us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:54.945106 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:54.945153 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:54.945242 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:54.950099 (+ 4857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:54.950103 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:54.950106 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:00:54.950124 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:54.950149 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:00:54.950159 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:54.950257 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:54.950489 (+ 232us) server_negotiation.cc:299] Negotiation successful
0811 19:00:54.950543 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260811 19:00:54.952229 24661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:54.938906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:46361 (local address 127.13.109.2:60905)
0811 19:00:54.943134 (+ 4228us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:54.943212 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:54.944832 (+ 1620us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:54.945503 (+ 671us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:54.945516 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:54.945652 (+ 136us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:54.949460 (+ 3808us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 19:00:54.949464 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:54.949557 (+ 93us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 19:00:54.949710 (+ 153us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:00:54.950271 (+ 561us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:00:54.950286 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:00:54.950297 (+ 11us) client_negotiation.cc:769] Sending connection context
0811 19:00:54.950540 (+ 243us) client_negotiation.cc:240] Negotiation successful
0811 19:00:54.950920 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":158,"threads_started":1}
I20260811 19:00:54.953651 24659 heartbeater.cc:347] Connected to a master server at 127.13.109.62:46361
I20260811 19:00:54.953948 24659 heartbeater.cc:464] Registering TS with master...
I20260811 19:00:54.954440 24659 heartbeater.cc:511] Master 127.13.109.62:46361 requested a full tablet report, sending...
I20260811 19:00:54.955655 24298 ts_manager.cc:194] Registered new tserver with Master: 8559057a68df4fee8648cb7eb058e674 (127.13.109.2:45731)
W20260811 19:00:55.066212 24662 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:00:55.066521 24662 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:00:55.066565 24662 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:00:55.070856 24662 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:00:55.070928 24662 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:00:55.070991 24662 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.3
I20260811 19:00:55.076179 24662 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:38177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.109.3
--webserver_port=0
--tserver_master_addrs=127.13.109.62:46361
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24662
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.3
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:00:55.077435 24662 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:00:55.078929 24662 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:00:55.089171 24670 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:55.089176 24667 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:55.089279 24662 server_base.cc:1034] running on GCE node
W20260811 19:00:55.089176 24668 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:55.089967 24662 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:00:55.090657 24662 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:00:55.091845 24662 hybrid_clock.cc:648] HybridClock initialized: now 1786474855091826 us; error 30 us; skew 500 ppm
I20260811 19:00:55.093726 24662 webserver.cc:460] Webserver started at http://127.13.109.3:39269/ using document root <none> and password file <none>
I20260811 19:00:55.094376 24662 fs_manager.cc:359] Metadata directory not provided
I20260811 19:00:55.094472 24662 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:00:55.094715 24662 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:00:55.096940 24662 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-2/data/instance:
uuid: "864326a6475a4def9c34ab673f481194"
format_stamp: "Formatted at 2026-08-11 19:00:55 on dist-test-slave-0kls"
server_key: "842886d215d0de393fa4ffb1d9170c6c"
server_key_iv: "08cd868998a8cd55f04922e5b8ca259f"
server_key_version: "encryptionkey@0"
I20260811 19:00:55.097565 24662 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-2/wal/instance:
uuid: "864326a6475a4def9c34ab673f481194"
format_stamp: "Formatted at 2026-08-11 19:00:55 on dist-test-slave-0kls"
server_key: "842886d215d0de393fa4ffb1d9170c6c"
server_key_iv: "08cd868998a8cd55f04922e5b8ca259f"
server_key_version: "encryptionkey@0"
I20260811 19:00:55.101637 24662 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260811 19:00:55.104519 24676 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:55.105924 24662 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 19:00:55.106058 24662 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-2/wal
uuid: "864326a6475a4def9c34ab673f481194"
format_stamp: "Formatted at 2026-08-11 19:00:55 on dist-test-slave-0kls"
server_key: "842886d215d0de393fa4ffb1d9170c6c"
server_key_iv: "08cd868998a8cd55f04922e5b8ca259f"
server_key_version: "encryptionkey@0"
I20260811 19:00:55.106191 24662 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:00:55.111436 24662 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:00:55.112210 24662 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:00:55.112394 24662 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:00:55.113006 24662 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:00:55.114046 24662 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:00:55.114111 24662 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:55.114174 24662 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:00:55.114210 24662 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:55.123454 24662 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.3:38011
I20260811 19:00:55.123479 24800 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.3:38011 every 8 connection(s)
I20260811 19:00:55.124493 24662 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
I20260811 19:00:55.130964 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 24662
I20260811 19:00:55.131124 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474792496492-13748-0/minicluster-data/ts-2/wal/instance
I20260811 19:00:55.131116 24345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:55.126504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.3:33805 (local address 127.13.109.62:46361)
0811 19:00:55.126660 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:55.126664 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:55.127765 (+ 1101us) server_negotiation.cc:407] Connection header received
0811 19:00:55.129161 (+ 1396us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:55.129164 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:55.129215 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:55.129307 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:55.130574 (+ 1267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:55.130578 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:55.130581 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:00:55.130596 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:55.130625 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:00:55.130634 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:55.130759 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:55.130907 (+ 148us) server_negotiation.cc:299] Negotiation successful
0811 19:00:55.130961 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 19:00:55.131470 13748 external_mini_cluster.cc:1428] Setting key ae02acf83ffaf413158ed59bf33d2646
I20260811 19:00:55.132216 24803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:55.126920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:46361 (local address 127.13.109.3:33805)
0811 19:00:55.127604 (+ 684us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:55.127646 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:55.128913 (+ 1267us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:55.129493 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:55.129501 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:55.129602 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:55.130286 (+ 684us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 19:00:55.130288 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:55.130333 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 19:00:55.130409 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:00:55.130746 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:00:55.130756 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:00:55.130763 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 19:00:55.130952 (+ 189us) client_negotiation.cc:240] Negotiation successful
0811 19:00:55.131220 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":124,"threads_started":1}
I20260811 19:00:55.133409 24801 heartbeater.cc:347] Connected to a master server at 127.13.109.62:46361
I20260811 19:00:55.133713 24801 heartbeater.cc:464] Registering TS with master...
I20260811 19:00:55.134222 24801 heartbeater.cc:511] Master 127.13.109.62:46361 requested a full tablet report, sending...
I20260811 19:00:55.135299 24297 ts_manager.cc:194] Registered new tserver with Master: 864326a6475a4def9c34ab673f481194 (127.13.109.3:38011)
I20260811 19:00:55.146469 13748 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 19:00:55.149885 24345 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:00:55.148672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45454 (local address 127.13.109.62:46361)
0811 19:00:55.148826 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:55.148830 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:55.149059 (+ 229us) server_negotiation.cc:407] Connection header received
0811 19:00:55.149164 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:55.149166 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:55.149204 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:55.149282 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:55.149675 (+ 393us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45454: BlockingRecv error: recv got EOF from 127.0.0.1:45454 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 19:00:55.150153 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 24367
I20260811 19:00:55.160377 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 24518
I20260811 19:00:55.168772 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 24662
I20260811 19:00:55.176342 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 24261
2026-08-11T19:00:55Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (898 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:00:55 dist-test-slave-0kls krb5kdc[24813](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:00:55 dist-test-slave-0kls krb5kdc[24813](info): set up 2 sockets
Aug 11 19:00:55 dist-test-slave-0kls krb5kdc[24813](info): commencing operation
krb5kdc: starting...
W20260811 19:00:56.296420 13748 mini_kdc.cc:121] Time spent starting KDC: real 1.095s user 0.006s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/krb5kdc/test-user.keytab.
Aug 11 19:00:56 dist-test-slave-0kls krb5kdc[24813](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474856, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:00:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:00:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.109.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.109.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.109.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab.
Entry for principal kudu/127.13.109.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab.
I20260811 19:00:56.439451 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.62:35173
--webserver_interface=127.13.109.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab
--principal=kudu/127.13.109.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:33335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.109.62:35173
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:00:56.559672 24827 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:00:56.560056 24827 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:00:56.560153 24827 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:00:56.564417 24827 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:00:56.564536 24827 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:00:56.564574 24827 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:00:56.564605 24827 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:00:56.564641 24827 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:00:56.570363 24827 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:33335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.109.62:35173
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab
--principal=kudu/127.13.109.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.62:35173
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.109.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24827
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:00:56.571517 24827 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:00:56.572546 24827 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:00:56.581308 24833 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:56.581300 24834 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:56.581358 24827 server_base.cc:1034] running on GCE node
W20260811 19:00:56.581283 24837 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:56.582176 24827 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:00:56.583325 24827 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:00:56.584484 24827 hybrid_clock.cc:648] HybridClock initialized: now 1786474856584459 us; error 38 us; skew 500 ppm
Aug 11 19:00:56 dist-test-slave-0kls krb5kdc[24813](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474856, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:00:56.587447 24827 init.cc:377] Logged in from keytab as kudu/127.13.109.62@KRBTEST.COM (short username kudu)
I20260811 19:00:56.588436 24827 webserver.cc:460] Webserver started at http://127.13.109.62:39369/ using document root <none> and password file <none>
I20260811 19:00:56.589032 24827 fs_manager.cc:359] Metadata directory not provided
I20260811 19:00:56.589098 24827 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:00:56.589349 24827 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:00:56.591634 24827 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/minicluster-data/master-0/data/instance:
uuid: "4dd67887d86a4ff3b4581124b628a0bc"
format_stamp: "Formatted at 2026-08-11 19:00:56 on dist-test-slave-0kls"
server_key: "79151ab9a8553faae5e6aac214e6a668"
server_key_iv: "ec518beebdfff5f28a65a6c7e19f2a33"
server_key_version: "encryptionkey@0"
I20260811 19:00:56.592243 24827 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/minicluster-data/master-0/wal/instance:
uuid: "4dd67887d86a4ff3b4581124b628a0bc"
format_stamp: "Formatted at 2026-08-11 19:00:56 on dist-test-slave-0kls"
server_key: "79151ab9a8553faae5e6aac214e6a668"
server_key_iv: "ec518beebdfff5f28a65a6c7e19f2a33"
server_key_version: "encryptionkey@0"
I20260811 19:00:56.596282 24827 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260811 19:00:56.598945 24844 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:56.600267 24827 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 19:00:56.600419 24827 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/minicluster-data/master-0/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/minicluster-data/master-0/wal
uuid: "4dd67887d86a4ff3b4581124b628a0bc"
format_stamp: "Formatted at 2026-08-11 19:00:56 on dist-test-slave-0kls"
server_key: "79151ab9a8553faae5e6aac214e6a668"
server_key_iv: "ec518beebdfff5f28a65a6c7e19f2a33"
server_key_version: "encryptionkey@0"
I20260811 19:00:56.600543 24827 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:00:56.621124 24827 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:00:56.624761 24827 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:00:56.624990 24827 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:00:56.633644 24827 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.62:35173
I20260811 19:00:56.634055 24900 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.62:35173 every 8 connection(s)
I20260811 19:00:56.634824 24827 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
I20260811 19:00:56.637835 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 24827
I20260811 19:00:56.637974 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/minicluster-data/master-0/wal/instance
I20260811 19:00:56.638010 24901 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:00:56.638300 13748 external_mini_cluster.cc:1428] Setting key 533f3093827f1580cfcc80e83ecc8c42
I20260811 19:00:56.645620 24901 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4dd67887d86a4ff3b4581124b628a0bc: Bootstrap starting.
Aug 11 19:00:56 dist-test-slave-0kls krb5kdc[24813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474856, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
I20260811 19:00:56.648615 24901 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4dd67887d86a4ff3b4581124b628a0bc: Neither blocks nor log segments found. Creating new log.
I20260811 19:00:56.649641 24901 log.cc:824] T 00000000000000000000000000000000 P 4dd67887d86a4ff3b4581124b628a0bc: Log is configured to *not* fsync() on all Append() calls
I20260811 19:00:56.652114 24901 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4dd67887d86a4ff3b4581124b628a0bc: No bootstrap required, opened a new log
I20260811 19:00:56.653388 24904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:56.639772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55282 (local address 127.13.109.62:35173)
0811 19:00:56.640211 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:56.640228 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:56.640267 (+ 39us) server_negotiation.cc:407] Connection header received
0811 19:00:56.640905 (+ 638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:56.640922 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:56.641290 (+ 368us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:56.641582 (+ 292us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:56.642575 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:56.643595 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:56.644794 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:56.645273 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:56.645388 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:56.648190 (+ 2802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:56.648196 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:56.648214 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:56.648250 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:56.650283 (+ 2033us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:56.650758 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:56.650764 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:56.650769 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:56.650906 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:56.651160 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:56.651163 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:56.651165 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:56.651698 (+ 533us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:56.651895 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:56.652098 (+ 203us) server_negotiation.cc:299] Negotiation successful
0811 19:00:56.652411 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":141,"threads_started":1}
I20260811 19:00:56.656451 24901 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4dd67887d86a4ff3b4581124b628a0bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4dd67887d86a4ff3b4581124b628a0bc" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 35173 } }
I20260811 19:00:56.656673 24901 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4dd67887d86a4ff3b4581124b628a0bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:00:56.656719 24901 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4dd67887d86a4ff3b4581124b628a0bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dd67887d86a4ff3b4581124b628a0bc, State: Initialized, Role: FOLLOWER
I20260811 19:00:56.657138 24901 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4dd67887d86a4ff3b4581124b628a0bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4dd67887d86a4ff3b4581124b628a0bc" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 35173 } }
I20260811 19:00:56.657263 24901 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4dd67887d86a4ff3b4581124b628a0bc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:00:56.657310 24901 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4dd67887d86a4ff3b4581124b628a0bc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:00:56.657383 24901 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4dd67887d86a4ff3b4581124b628a0bc [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:00:56.658387 24901 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4dd67887d86a4ff3b4581124b628a0bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4dd67887d86a4ff3b4581124b628a0bc" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 35173 } }
I20260811 19:00:56.658694 24901 leader_election.cc:302] T 00000000000000000000000000000000 P 4dd67887d86a4ff3b4581124b628a0bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4dd67887d86a4ff3b4581124b628a0bc; no voters:
I20260811 19:00:56.658965 24901 leader_election.cc:288] T 00000000000000000000000000000000 P 4dd67887d86a4ff3b4581124b628a0bc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:00:56.659121 24906 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4dd67887d86a4ff3b4581124b628a0bc [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:00:56.659356 24906 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4dd67887d86a4ff3b4581124b628a0bc [term 1 LEADER]: Becoming Leader. State: Replica: 4dd67887d86a4ff3b4581124b628a0bc, State: Running, Role: LEADER
I20260811 19:00:56.659709 24906 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4dd67887d86a4ff3b4581124b628a0bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4dd67887d86a4ff3b4581124b628a0bc" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 35173 } }
I20260811 19:00:56.660271 24901 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4dd67887d86a4ff3b4581124b628a0bc [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:00:56.661449 24908 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4dd67887d86a4ff3b4581124b628a0bc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4dd67887d86a4ff3b4581124b628a0bc. Latest consensus state: current_term: 1 leader_uuid: "4dd67887d86a4ff3b4581124b628a0bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4dd67887d86a4ff3b4581124b628a0bc" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 35173 } } }
I20260811 19:00:56.661592 24908 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4dd67887d86a4ff3b4581124b628a0bc [sys.catalog]: This master's current role is: LEADER
I20260811 19:00:56.662024 24917 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:00:56.661979 24907 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4dd67887d86a4ff3b4581124b628a0bc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4dd67887d86a4ff3b4581124b628a0bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4dd67887d86a4ff3b4581124b628a0bc" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 35173 } } }
I20260811 19:00:56.662142 24907 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4dd67887d86a4ff3b4581124b628a0bc [sys.catalog]: This master's current role is: LEADER
I20260811 19:00:56.665544 24917 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:00:56.671037 24917 catalog_manager.cc:1347] Generated new cluster ID: d7f3bf895e024b2f9683dff05b7860ee
I20260811 19:00:56.671108 24917 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:00:56.677017 24917 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:00:56.678202 24917 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:00:56.692227 24917 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4dd67887d86a4ff3b4581124b628a0bc: Generated new TSK 0
I20260811 19:00:56.693020 24917 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.109.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.109.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.109.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.1.keytab.
Entry for principal kudu/127.13.109.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.1.keytab.
I20260811 19:00:56.733662 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.1:0
--local_ip_for_outbound_sockets=127.13.109.1
--webserver_interface=127.13.109.1
--webserver_port=0
--tserver_master_addrs=127.13.109.62:35173
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.1.keytab
--principal=kudu/127.13.109.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:33335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:00:56.876590 24929 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:00:56.876880 24929 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:00:56.876927 24929 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:00:56.881600 24929 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:00:56.881683 24929 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:00:56.881752 24929 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.1
I20260811 19:00:56.887408 24929 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:33335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.1.keytab
--principal=kudu/127.13.109.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.109.1
--webserver_port=0
--tserver_master_addrs=127.13.109.62:35173
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24929
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.1
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:00:56.888621 24929 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:00:56.889683 24929 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:00:56.900053 24938 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:56.900053 24934 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:56.900056 24935 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:56.900113 24929 server_base.cc:1034] running on GCE node
I20260811 19:00:56.900759 24929 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:00:56.901528 24929 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:00:56.902726 24929 hybrid_clock.cc:648] HybridClock initialized: now 1786474856902702 us; error 36 us; skew 500 ppm
Aug 11 19:00:56 dist-test-slave-0kls krb5kdc[24813](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474856, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:00:56.905824 24929 init.cc:377] Logged in from keytab as kudu/127.13.109.1@KRBTEST.COM (short username kudu)
I20260811 19:00:56.906818 24929 webserver.cc:460] Webserver started at http://127.13.109.1:38997/ using document root <none> and password file <none>
I20260811 19:00:56.907423 24929 fs_manager.cc:359] Metadata directory not provided
I20260811 19:00:56.907516 24929 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:00:56.907769 24929 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:00:56.910079 24929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/minicluster-data/ts-0/data/instance:
uuid: "e49407102e8d42ea98655358ce020b88"
format_stamp: "Formatted at 2026-08-11 19:00:56 on dist-test-slave-0kls"
server_key: "7a843e9580b601bc257694547a8e3ed1"
server_key_iv: "83952df77c3f771a044e6eb53905e81d"
server_key_version: "encryptionkey@0"
I20260811 19:00:56.910696 24929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/minicluster-data/ts-0/wal/instance:
uuid: "e49407102e8d42ea98655358ce020b88"
format_stamp: "Formatted at 2026-08-11 19:00:56 on dist-test-slave-0kls"
server_key: "7a843e9580b601bc257694547a8e3ed1"
server_key_iv: "83952df77c3f771a044e6eb53905e81d"
server_key_version: "encryptionkey@0"
I20260811 19:00:56.915081 24929 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260811 19:00:56.917908 24945 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:56.919173 24929 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260811 19:00:56.919315 24929 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/minicluster-data/ts-0/wal
uuid: "e49407102e8d42ea98655358ce020b88"
format_stamp: "Formatted at 2026-08-11 19:00:56 on dist-test-slave-0kls"
server_key: "7a843e9580b601bc257694547a8e3ed1"
server_key_iv: "83952df77c3f771a044e6eb53905e81d"
server_key_version: "encryptionkey@0"
I20260811 19:00:56.919440 24929 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:00:56.945529 24929 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:00:56.948760 24929 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:00:56.949007 24929 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:00:56.949709 24929 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:00:56.950686 24929 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:00:56.950767 24929 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:56.950855 24929 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:00:56.950896 24929 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:56.960781 24929 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.1:34779
I20260811 19:00:56.960808 25065 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.1:34779 every 8 connection(s)
I20260811 19:00:56.961987 24929 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
I20260811 19:00:56.963757 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 24929
I20260811 19:00:56.963897 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/minicluster-data/ts-0/wal/instance
I20260811 19:00:56.964237 13748 external_mini_cluster.cc:1428] Setting key 50ae14bfaa9c2b960f5cbe7e50a414fb
Aug 11 19:00:56 dist-test-slave-0kls krb5kdc[24813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474856, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.1@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.109.2@KRBTEST.COM; defaulting to no policy
I20260811 19:00:56.977674 24904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:56.964542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.1:53337 (local address 127.13.109.62:35173)
0811 19:00:56.964689 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:56.964698 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:56.965566 (+ 868us) server_negotiation.cc:407] Connection header received
0811 19:00:56.967120 (+ 1554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:56.967123 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:56.967181 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:56.967286 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:56.968570 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:56.969281 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:56.970547 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:56.970984 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:56.971123 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:56.973706 (+ 2583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:56.973710 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:56.973715 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:56.973753 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:56.975403 (+ 1650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:56.975933 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:56.975936 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:56.975937 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:56.976017 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:56.976404 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:56.976407 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:56.976408 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:56.976640 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:56.976732 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:56.977310 (+ 578us) server_negotiation.cc:299] Negotiation successful
0811 19:00:56.977439 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.109.2@KRBTEST.COM" created.
I20260811 19:00:56.978888 25068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:56.964818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:35173 (local address 127.13.109.1:53337)
0811 19:00:56.965394 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:56.965456 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:56.966879 (+ 1423us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:56.967486 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:56.967494 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:56.967885 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:56.968408 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:56.968422 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:56.969420 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:56.969423 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:56.970408 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:56.970419 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:56.971158 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:56.971161 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:56.971326 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:56.971962 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:56.971980 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:56.973503 (+ 1523us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:00:56.975523 (+ 2020us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:56.975529 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:56.975544 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:56.975821 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:56.976127 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:56.976130 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:56.976131 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:56.976295 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:56.976748 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:00:56.976753 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:00:56.977070 (+ 317us) client_negotiation.cc:769] Sending connection context
0811 19:00:56.977258 (+ 188us) client_negotiation.cc:240] Negotiation successful
0811 19:00:56.977546 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":165,"threads_started":1}
I20260811 19:00:56.980373 25066 heartbeater.cc:347] Connected to a master server at 127.13.109.62:35173
I20260811 19:00:56.980654 25066 heartbeater.cc:464] Registering TS with master...
I20260811 19:00:56.981200 25066 heartbeater.cc:511] Master 127.13.109.62:35173 requested a full tablet report, sending...
I20260811 19:00:56.982856 24862 ts_manager.cc:194] Registered new tserver with Master: e49407102e8d42ea98655358ce020b88 (127.13.109.1:34779)
I20260811 19:00:56.984047 24862 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.109.1@KRBTEST.COM'} at 127.13.109.1:53337
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.109.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.2.keytab.
Entry for principal kudu/127.13.109.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.2.keytab.
I20260811 19:00:56.996917 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.2:0
--local_ip_for_outbound_sockets=127.13.109.2
--webserver_interface=127.13.109.2
--webserver_port=0
--tserver_master_addrs=127.13.109.62:35173
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.2.keytab
--principal=kudu/127.13.109.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:33335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:00:57.117100 25071 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:00:57.117425 25071 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:00:57.117516 25071 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:00:57.121735 25071 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:00:57.121868 25071 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:00:57.121961 25071 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.2
I20260811 19:00:57.127681 25071 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:33335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.2.keytab
--principal=kudu/127.13.109.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.109.2
--webserver_port=0
--tserver_master_addrs=127.13.109.62:35173
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25071
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.2
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:00:57.128865 25071 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:00:57.129982 25071 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:00:57.140028 25078 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:57.140028 25080 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:57.140029 25076 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:57.140103 25071 server_base.cc:1034] running on GCE node
I20260811 19:00:57.140820 25071 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:00:57.141525 25071 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:00:57.142715 25071 hybrid_clock.cc:648] HybridClock initialized: now 1786474857142696 us; error 35 us; skew 500 ppm
Aug 11 19:00:57 dist-test-slave-0kls krb5kdc[24813](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474857, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:00:57.145493 25071 init.cc:377] Logged in from keytab as kudu/127.13.109.2@KRBTEST.COM (short username kudu)
I20260811 19:00:57.146510 25071 webserver.cc:460] Webserver started at http://127.13.109.2:40139/ using document root <none> and password file <none>
I20260811 19:00:57.147133 25071 fs_manager.cc:359] Metadata directory not provided
I20260811 19:00:57.147228 25071 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:00:57.147468 25071 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:00:57.149614 25071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/minicluster-data/ts-1/data/instance:
uuid: "222dca4ec73d42448a7ae1520cb16f8f"
format_stamp: "Formatted at 2026-08-11 19:00:57 on dist-test-slave-0kls"
server_key: "a2ce08366c2fe8e37ed4bed10535ff18"
server_key_iv: "ae7077bbd48dd8efd2c6ba63b132d8c3"
server_key_version: "encryptionkey@0"
I20260811 19:00:57.150252 25071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/minicluster-data/ts-1/wal/instance:
uuid: "222dca4ec73d42448a7ae1520cb16f8f"
format_stamp: "Formatted at 2026-08-11 19:00:57 on dist-test-slave-0kls"
server_key: "a2ce08366c2fe8e37ed4bed10535ff18"
server_key_iv: "ae7077bbd48dd8efd2c6ba63b132d8c3"
server_key_version: "encryptionkey@0"
I20260811 19:00:57.154363 25071 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 19:00:57.156987 25088 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:57.158356 25071 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 19:00:57.158532 25071 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/minicluster-data/ts-1/wal
uuid: "222dca4ec73d42448a7ae1520cb16f8f"
format_stamp: "Formatted at 2026-08-11 19:00:57 on dist-test-slave-0kls"
server_key: "a2ce08366c2fe8e37ed4bed10535ff18"
server_key_iv: "ae7077bbd48dd8efd2c6ba63b132d8c3"
server_key_version: "encryptionkey@0"
I20260811 19:00:57.158672 25071 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:00:57.169499 25071 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:00:57.172461 25071 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:00:57.172677 25071 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:00:57.173321 25071 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:00:57.174292 25071 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:00:57.174360 25071 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:57.174430 25071 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:00:57.174474 25071 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:57.183817 25071 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.2:46181
I20260811 19:00:57.183836 25212 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.2:46181 every 8 connection(s)
I20260811 19:00:57.184762 25071 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
Aug 11 19:00:57 dist-test-slave-0kls krb5kdc[24813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474857, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.2@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
I20260811 19:00:57.194658 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 25071
I20260811 19:00:57.194787 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/minicluster-data/ts-1/wal/instance
I20260811 19:00:57.195160 13748 external_mini_cluster.cc:1428] Setting key 88e4221c4605c2c954fe94fb2f1fd532
I20260811 19:00:57.198886 24904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:57.186672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.2:55995 (local address 127.13.109.62:35173)
0811 19:00:57.186818 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:57.186822 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:57.187639 (+ 817us) server_negotiation.cc:407] Connection header received
0811 19:00:57.188976 (+ 1337us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:57.188979 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:57.189026 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:57.189103 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:57.190350 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:57.190910 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:57.192032 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:57.192309 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:57.192459 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:57.194971 (+ 2512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:57.194976 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:57.194979 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:57.195014 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:57.196634 (+ 1620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:57.197135 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:57.197138 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:57.197142 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:57.197242 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:57.197567 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:57.197573 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:57.197577 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:57.197887 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:57.197995 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:57.198489 (+ 494us) server_negotiation.cc:299] Negotiation successful
0811 19:00:57.198640 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 19:00:57.199908 25215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:57.187043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:35173 (local address 127.13.109.2:55995)
0811 19:00:57.187490 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:57.187533 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:57.188776 (+ 1243us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:57.189291 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:57.189298 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:57.189650 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:57.190209 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:57.190223 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:57.191028 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:57.191030 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:57.191897 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:57.191906 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:57.192473 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:57.192476 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:57.192616 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:57.193220 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:57.193237 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:57.194766 (+ 1529us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:00:57.196741 (+ 1975us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:57.196749 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:57.196765 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:57.197034 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:57.197340 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:57.197342 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:57.197344 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:57.197479 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:57.197989 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:00:57.197994 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:00:57.198246 (+ 252us) client_negotiation.cc:769] Sending connection context
0811 19:00:57.198412 (+ 166us) client_negotiation.cc:240] Negotiation successful
0811 19:00:57.198687 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":108,"threads_started":1}
I20260811 19:00:57.201124 25213 heartbeater.cc:347] Connected to a master server at 127.13.109.62:35173
I20260811 19:00:57.201378 25213 heartbeater.cc:464] Registering TS with master...
I20260811 19:00:57.201963 25213 heartbeater.cc:511] Master 127.13.109.62:35173 requested a full tablet report, sending...
I20260811 19:00:57.203083 24862 ts_manager.cc:194] Registered new tserver with Master: 222dca4ec73d42448a7ae1520cb16f8f (127.13.109.2:46181)
I20260811 19:00:57.203764 24862 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.109.2@KRBTEST.COM'} at 127.13.109.2:55995
WARNING: no policy specified for kudu/127.13.109.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.109.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.109.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.3.keytab.
Entry for principal kudu/127.13.109.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.3.keytab.
I20260811 19:00:57.225149 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.3:0
--local_ip_for_outbound_sockets=127.13.109.3
--webserver_interface=127.13.109.3
--webserver_port=0
--tserver_master_addrs=127.13.109.62:35173
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.3.keytab
--principal=kudu/127.13.109.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:33335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:00:57.345324 25219 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:00:57.345639 25219 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:00:57.345687 25219 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:00:57.349995 25219 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:00:57.350073 25219 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:00:57.350135 25219 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.3
I20260811 19:00:57.355811 25219 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:33335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.3.keytab
--principal=kudu/127.13.109.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.109.3
--webserver_port=0
--tserver_master_addrs=127.13.109.62:35173
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25219
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.3
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:00:57.357005 25219 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:00:57.358160 25219 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:00:57.368889 25228 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:57.368891 25224 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:57.368955 25225 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:57.369140 25219 server_base.cc:1034] running on GCE node
I20260811 19:00:57.369740 25219 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:00:57.370512 25219 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:00:57.371672 25219 hybrid_clock.cc:648] HybridClock initialized: now 1786474857371655 us; error 30 us; skew 500 ppm
Aug 11 19:00:57 dist-test-slave-0kls krb5kdc[24813](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474857, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:00:57.374480 25219 init.cc:377] Logged in from keytab as kudu/127.13.109.3@KRBTEST.COM (short username kudu)
I20260811 19:00:57.375454 25219 webserver.cc:460] Webserver started at http://127.13.109.3:45125/ using document root <none> and password file <none>
I20260811 19:00:57.376041 25219 fs_manager.cc:359] Metadata directory not provided
I20260811 19:00:57.376103 25219 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:00:57.376355 25219 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:00:57.378712 25219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/minicluster-data/ts-2/data/instance:
uuid: "05898439786a4076b4704825521128bc"
format_stamp: "Formatted at 2026-08-11 19:00:57 on dist-test-slave-0kls"
server_key: "93a1ab74efb7a475fb665e84f7f037b8"
server_key_iv: "3065cd6375ab4b5cb3afdbd6112d14d4"
server_key_version: "encryptionkey@0"
I20260811 19:00:57.379298 25219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/minicluster-data/ts-2/wal/instance:
uuid: "05898439786a4076b4704825521128bc"
format_stamp: "Formatted at 2026-08-11 19:00:57 on dist-test-slave-0kls"
server_key: "93a1ab74efb7a475fb665e84f7f037b8"
server_key_iv: "3065cd6375ab4b5cb3afdbd6112d14d4"
server_key_version: "encryptionkey@0"
I20260811 19:00:57.383190 25219 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 19:00:57.385960 25235 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:57.387296 25219 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 19:00:57.387426 25219 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/minicluster-data/ts-2/wal
uuid: "05898439786a4076b4704825521128bc"
format_stamp: "Formatted at 2026-08-11 19:00:57 on dist-test-slave-0kls"
server_key: "93a1ab74efb7a475fb665e84f7f037b8"
server_key_iv: "3065cd6375ab4b5cb3afdbd6112d14d4"
server_key_version: "encryptionkey@0"
I20260811 19:00:57.387548 25219 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:00:57.402081 25219 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:00:57.405382 25219 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:00:57.405584 25219 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:00:57.406391 25219 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:00:57.407285 25219 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:00:57.407333 25219 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:57.407400 25219 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:00:57.407434 25219 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:57.416603 25219 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.3:38259
I20260811 19:00:57.416636 25364 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.3:38259 every 8 connection(s)
I20260811 19:00:57.417655 25219 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
I20260811 19:00:57.422827 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 25219
I20260811 19:00:57.422971 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/minicluster-data/ts-2/wal/instance
I20260811 19:00:57.423282 13748 external_mini_cluster.cc:1428] Setting key b98b815ec59d8e5fd14c74aeddda1d92
Aug 11 19:00:57 dist-test-slave-0kls krb5kdc[24813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474857, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.3@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
I20260811 19:00:57.431854 24904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:57.419517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.3:59673 (local address 127.13.109.62:35173)
0811 19:00:57.419668 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:57.419672 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:57.420718 (+ 1046us) server_negotiation.cc:407] Connection header received
0811 19:00:57.421663 (+ 945us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:57.421667 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:57.421728 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:57.421886 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:57.423268 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:57.424045 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:57.425423 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:57.425637 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:57.425747 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:57.428358 (+ 2611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:57.428361 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:57.428364 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:57.428390 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:57.429667 (+ 1277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:57.430255 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:57.430259 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:57.430261 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:57.430347 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:57.430702 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:57.430706 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:57.430711 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:57.430910 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:57.431022 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:57.431549 (+ 527us) server_negotiation.cc:299] Negotiation successful
0811 19:00:57.431682 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 19:00:57.432753 25368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:57.419923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:35173 (local address 127.13.109.3:59673)
0811 19:00:57.420567 (+ 644us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:57.420612 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:57.421451 (+ 839us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:57.422061 (+ 610us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:57.422069 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:57.422440 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:57.423118 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:57.423136 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:57.424188 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:57.424191 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:57.425292 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:57.425303 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:57.425813 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:57.425819 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:57.426006 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:57.426588 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:57.426606 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:57.428169 (+ 1563us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:00:57.429828 (+ 1659us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:57.429835 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:57.429855 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:57.430139 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:57.430441 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:57.430445 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:57.430447 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:57.430607 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:57.431022 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:00:57.431030 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:00:57.431308 (+ 278us) client_negotiation.cc:769] Sending connection context
0811 19:00:57.431508 (+ 200us) client_negotiation.cc:240] Negotiation successful
0811 19:00:57.431882 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":113,"threads_started":1}
I20260811 19:00:57.433981 25365 heartbeater.cc:347] Connected to a master server at 127.13.109.62:35173
I20260811 19:00:57.434268 25365 heartbeater.cc:464] Registering TS with master...
I20260811 19:00:57.434824 25365 heartbeater.cc:511] Master 127.13.109.62:35173 requested a full tablet report, sending...
I20260811 19:00:57.435945 24862 ts_manager.cc:194] Registered new tserver with Master: 05898439786a4076b4704825521128bc (127.13.109.3:38259)
I20260811 19:00:57.436527 24862 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.109.3@KRBTEST.COM'} at 127.13.109.3:59673
I20260811 19:00:57.437119 13748 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 19:00:57.446139 24904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:57.439218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55292 (local address 127.13.109.62:35173)
0811 19:00:57.439338 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:57.439341 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:57.439531 (+ 190us) server_negotiation.cc:407] Connection header received
0811 19:00:57.439651 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:57.439654 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:57.439691 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:57.439761 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:57.440403 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:57.440895 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:57.441764 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:57.442025 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:57.442102 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:57.443491 (+ 1389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:57.443494 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:57.443495 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:57.443513 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:57.444681 (+ 1168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:57.445058 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:57.445061 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:57.445062 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:57.445133 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:57.445366 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:57.445368 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:57.445369 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:57.445569 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:57.445671 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:57.445921 (+ 250us) server_negotiation.cc:299] Negotiation successful
0811 19:00:57.446020 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 19:00:57.449177 24860 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55292:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 19:00:57.452227 24860 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 19:00:57.467324 25378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:57.461120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.3:38259 (local address 127.0.0.1:46164)
0811 19:00:57.461596 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:57.461611 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:57.461710 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:57.462970 (+ 1260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:57.462973 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:57.462993 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:00:57.463150 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:57.463157 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:57.464774 (+ 1617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:57.464777 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:57.465873 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:57.465882 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:57.466798 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:57.466802 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:57.466930 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:57.466957 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 19:00:57.467006 (+ 49us) client_negotiation.cc:240] Negotiation successful
0811 19:00:57.467170 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":398,"mutex_wait_us":22,"thread_start_us":125,"threads_started":1}
I20260811 19:00:57.467334 25379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:57.461165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46120 (local address 127.13.109.1:34779)
0811 19:00:57.462013 (+ 848us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:57.462030 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:57.462059 (+ 29us) server_negotiation.cc:407] Connection header received
0811 19:00:57.462138 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:57.462144 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:57.462354 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:57.462518 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:00:57.462888 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:57.463927 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:57.465674 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:57.466581 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:57.466694 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:57.466858 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:57.467083 (+ 225us) server_negotiation.cc:299] Negotiation successful
0811 19:00:57.467181 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":740,"thread_start_us":120,"threads_started":1}
I20260811 19:00:57.467624 25380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:57.461963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46164 (local address 127.13.109.3:38259)
0811 19:00:57.462482 (+ 519us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:57.462498 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:57.462524 (+ 26us) server_negotiation.cc:407] Connection header received
0811 19:00:57.462602 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:57.462607 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:57.462808 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:57.463314 (+ 506us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:00:57.463470 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:57.464602 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:57.465997 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:57.466653 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:57.467140 (+ 487us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:57.467291 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:57.467347 (+ 56us) server_negotiation.cc:299] Negotiation successful
0811 19:00:57.467442 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":162,"threads_started":1}
I20260811 19:00:57.468336 25377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:57.461025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.1:34779 (local address 127.0.0.1:46120)
0811 19:00:57.461508 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:57.461557 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:57.461682 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:57.462582 (+ 900us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:57.462588 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:57.462607 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:00:57.462747 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:57.462754 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:57.464072 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:57.464075 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:57.465533 (+ 1458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:57.465544 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:57.466706 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:57.466709 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:57.466857 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:57.466911 (+ 54us) client_negotiation.cc:769] Sending connection context
0811 19:00:57.467039 (+ 128us) client_negotiation.cc:240] Negotiation successful
0811 19:00:57.467568 (+ 529us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"mutex_wait_us":30,"thread_start_us":127,"threads_started":1}
I20260811 19:00:57.468976 25381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:57.462607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.2:46181 (local address 127.0.0.1:48734)
0811 19:00:57.463003 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:57.463017 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:57.463097 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:57.463827 (+ 730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:57.463830 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:57.463849 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:00:57.464001 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:57.464011 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:57.465251 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:57.465257 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:57.466323 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:57.466335 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:57.468311 (+ 1976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:57.468316 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:57.468684 (+ 368us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:57.468703 (+ 19us) client_negotiation.cc:769] Sending connection context
0811 19:00:57.468787 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 19:00:57.468859 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":106,"threads_started":1}
I20260811 19:00:57.469075 25382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:57.462951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48734 (local address 127.13.109.2:46181)
0811 19:00:57.463399 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:57.463410 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:57.463433 (+ 23us) server_negotiation.cc:407] Connection header received
0811 19:00:57.463504 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:57.463509 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:57.463684 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:57.463834 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:00:57.464126 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:57.465114 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:57.466478 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:57.467282 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:57.467385 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:57.468079 (+ 694us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:57.468825 (+ 746us) server_negotiation.cc:299] Negotiation successful
0811 19:00:57.468918 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":117,"threads_started":1}
I20260811 19:00:57.470886 25290 tablet_service.cc:1467] Processing CreateTablet for tablet 77a9013d2d9e44c982d238de3d469850 (DEFAULT_TABLE table=test-table [id=6d54c8ba3d3b4821afefad12ad690a09]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:00:57.471127 24998 tablet_service.cc:1467] Processing CreateTablet for tablet 77a9013d2d9e44c982d238de3d469850 (DEFAULT_TABLE table=test-table [id=6d54c8ba3d3b4821afefad12ad690a09]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:00:57.471388 25143 tablet_service.cc:1467] Processing CreateTablet for tablet 77a9013d2d9e44c982d238de3d469850 (DEFAULT_TABLE table=test-table [id=6d54c8ba3d3b4821afefad12ad690a09]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:00:57.472162 24998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77a9013d2d9e44c982d238de3d469850. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:00:57.472167 25290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77a9013d2d9e44c982d238de3d469850. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:00:57.472437 25143 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77a9013d2d9e44c982d238de3d469850. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:00:57.478914 25383 tablet_bootstrap.cc:492] T 77a9013d2d9e44c982d238de3d469850 P 222dca4ec73d42448a7ae1520cb16f8f: Bootstrap starting.
I20260811 19:00:57.480597 25385 tablet_bootstrap.cc:492] T 77a9013d2d9e44c982d238de3d469850 P e49407102e8d42ea98655358ce020b88: Bootstrap starting.
I20260811 19:00:57.481699 25383 tablet_bootstrap.cc:654] T 77a9013d2d9e44c982d238de3d469850 P 222dca4ec73d42448a7ae1520cb16f8f: Neither blocks nor log segments found. Creating new log.
I20260811 19:00:57.482224 25384 tablet_bootstrap.cc:492] T 77a9013d2d9e44c982d238de3d469850 P 05898439786a4076b4704825521128bc: Bootstrap starting.
I20260811 19:00:57.482558 25383 log.cc:824] T 77a9013d2d9e44c982d238de3d469850 P 222dca4ec73d42448a7ae1520cb16f8f: Log is configured to *not* fsync() on all Append() calls
I20260811 19:00:57.482666 25385 tablet_bootstrap.cc:654] T 77a9013d2d9e44c982d238de3d469850 P e49407102e8d42ea98655358ce020b88: Neither blocks nor log segments found. Creating new log.
I20260811 19:00:57.483553 25385 log.cc:824] T 77a9013d2d9e44c982d238de3d469850 P e49407102e8d42ea98655358ce020b88: Log is configured to *not* fsync() on all Append() calls
I20260811 19:00:57.484510 25384 tablet_bootstrap.cc:654] T 77a9013d2d9e44c982d238de3d469850 P 05898439786a4076b4704825521128bc: Neither blocks nor log segments found. Creating new log.
I20260811 19:00:57.485363 25385 tablet_bootstrap.cc:492] T 77a9013d2d9e44c982d238de3d469850 P e49407102e8d42ea98655358ce020b88: No bootstrap required, opened a new log
I20260811 19:00:57.485558 25383 tablet_bootstrap.cc:492] T 77a9013d2d9e44c982d238de3d469850 P 222dca4ec73d42448a7ae1520cb16f8f: No bootstrap required, opened a new log
I20260811 19:00:57.485579 25385 ts_tablet_manager.cc:1397] T 77a9013d2d9e44c982d238de3d469850 P e49407102e8d42ea98655358ce020b88: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 19:00:57.485769 25383 ts_tablet_manager.cc:1397] T 77a9013d2d9e44c982d238de3d469850 P 222dca4ec73d42448a7ae1520cb16f8f: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260811 19:00:57.485822 25384 log.cc:824] T 77a9013d2d9e44c982d238de3d469850 P 05898439786a4076b4704825521128bc: Log is configured to *not* fsync() on all Append() calls
I20260811 19:00:57.488066 25384 tablet_bootstrap.cc:492] T 77a9013d2d9e44c982d238de3d469850 P 05898439786a4076b4704825521128bc: No bootstrap required, opened a new log
I20260811 19:00:57.488283 25384 ts_tablet_manager.cc:1397] T 77a9013d2d9e44c982d238de3d469850 P 05898439786a4076b4704825521128bc: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 19:00:57.489221 25385 raft_consensus.cc:348] T 77a9013d2d9e44c982d238de3d469850 P e49407102e8d42ea98655358ce020b88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e49407102e8d42ea98655358ce020b88" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 34779 } } peers { permanent_uuid: "05898439786a4076b4704825521128bc" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 38259 } } peers { permanent_uuid: "222dca4ec73d42448a7ae1520cb16f8f" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 46181 } }
I20260811 19:00:57.489475 25385 raft_consensus.cc:374] T 77a9013d2d9e44c982d238de3d469850 P e49407102e8d42ea98655358ce020b88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:00:57.489530 25385 raft_consensus.cc:729] T 77a9013d2d9e44c982d238de3d469850 P e49407102e8d42ea98655358ce020b88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e49407102e8d42ea98655358ce020b88, State: Initialized, Role: FOLLOWER
I20260811 19:00:57.489723 25383 raft_consensus.cc:348] T 77a9013d2d9e44c982d238de3d469850 P 222dca4ec73d42448a7ae1520cb16f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e49407102e8d42ea98655358ce020b88" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 34779 } } peers { permanent_uuid: "05898439786a4076b4704825521128bc" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 38259 } } peers { permanent_uuid: "222dca4ec73d42448a7ae1520cb16f8f" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 46181 } }
I20260811 19:00:57.490113 25383 raft_consensus.cc:374] T 77a9013d2d9e44c982d238de3d469850 P 222dca4ec73d42448a7ae1520cb16f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:00:57.490177 25383 raft_consensus.cc:729] T 77a9013d2d9e44c982d238de3d469850 P 222dca4ec73d42448a7ae1520cb16f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 222dca4ec73d42448a7ae1520cb16f8f, State: Initialized, Role: FOLLOWER
I20260811 19:00:57.490271 25385 consensus_queue.cc:260] T 77a9013d2d9e44c982d238de3d469850 P e49407102e8d42ea98655358ce020b88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e49407102e8d42ea98655358ce020b88" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 34779 } } peers { permanent_uuid: "05898439786a4076b4704825521128bc" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 38259 } } peers { permanent_uuid: "222dca4ec73d42448a7ae1520cb16f8f" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 46181 } }
I20260811 19:00:57.490644 25383 consensus_queue.cc:260] T 77a9013d2d9e44c982d238de3d469850 P 222dca4ec73d42448a7ae1520cb16f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e49407102e8d42ea98655358ce020b88" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 34779 } } peers { permanent_uuid: "05898439786a4076b4704825521128bc" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 38259 } } peers { permanent_uuid: "222dca4ec73d42448a7ae1520cb16f8f" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 46181 } }
I20260811 19:00:57.491545 25385 ts_tablet_manager.cc:1428] T 77a9013d2d9e44c982d238de3d469850 P e49407102e8d42ea98655358ce020b88: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 19:00:57.491607 25383 ts_tablet_manager.cc:1428] T 77a9013d2d9e44c982d238de3d469850 P 222dca4ec73d42448a7ae1520cb16f8f: Time spent starting tablet: real 0.006s user 0.000s sys 0.005s
I20260811 19:00:57.491948 25066 heartbeater.cc:503] Master 127.13.109.62:35173 was elected leader, sending a full tablet report...
I20260811 19:00:57.491958 25213 heartbeater.cc:503] Master 127.13.109.62:35173 was elected leader, sending a full tablet report...
I20260811 19:00:57.493888 25384 raft_consensus.cc:348] T 77a9013d2d9e44c982d238de3d469850 P 05898439786a4076b4704825521128bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e49407102e8d42ea98655358ce020b88" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 34779 } } peers { permanent_uuid: "05898439786a4076b4704825521128bc" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 38259 } } peers { permanent_uuid: "222dca4ec73d42448a7ae1520cb16f8f" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 46181 } }
I20260811 19:00:57.494161 25384 raft_consensus.cc:374] T 77a9013d2d9e44c982d238de3d469850 P 05898439786a4076b4704825521128bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:00:57.494225 25384 raft_consensus.cc:729] T 77a9013d2d9e44c982d238de3d469850 P 05898439786a4076b4704825521128bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05898439786a4076b4704825521128bc, State: Initialized, Role: FOLLOWER
I20260811 19:00:57.494753 25384 consensus_queue.cc:260] T 77a9013d2d9e44c982d238de3d469850 P 05898439786a4076b4704825521128bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e49407102e8d42ea98655358ce020b88" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 34779 } } peers { permanent_uuid: "05898439786a4076b4704825521128bc" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 38259 } } peers { permanent_uuid: "222dca4ec73d42448a7ae1520cb16f8f" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 46181 } }
I20260811 19:00:57.495828 25384 ts_tablet_manager.cc:1428] T 77a9013d2d9e44c982d238de3d469850 P 05898439786a4076b4704825521128bc: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260811 19:00:57.495914 25365 heartbeater.cc:503] Master 127.13.109.62:35173 was elected leader, sending a full tablet report...
W20260811 19:00:57.668732 25367 tablet.cc:2369] T 77a9013d2d9e44c982d238de3d469850 P 05898439786a4076b4704825521128bc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 19:00:57.686115 25214 tablet.cc:2369] T 77a9013d2d9e44c982d238de3d469850 P 222dca4ec73d42448a7ae1520cb16f8f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 19:00:57.713639 25067 tablet.cc:2369] T 77a9013d2d9e44c982d238de3d469850 P e49407102e8d42ea98655358ce020b88: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 19:00:57.721439 25389 raft_consensus.cc:482] T 77a9013d2d9e44c982d238de3d469850 P e49407102e8d42ea98655358ce020b88 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:00:57.721678 25389 raft_consensus.cc:504] T 77a9013d2d9e44c982d238de3d469850 P e49407102e8d42ea98655358ce020b88 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e49407102e8d42ea98655358ce020b88" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 34779 } } peers { permanent_uuid: "05898439786a4076b4704825521128bc" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 38259 } } peers { permanent_uuid: "222dca4ec73d42448a7ae1520cb16f8f" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 46181 } }
I20260811 19:00:57.722970 25389 leader_election.cc:288] T 77a9013d2d9e44c982d238de3d469850 P e49407102e8d42ea98655358ce020b88 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05898439786a4076b4704825521128bc (127.13.109.3:38259), 222dca4ec73d42448a7ae1520cb16f8f (127.13.109.2:46181)
I20260811 19:00:57.727257 25380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:57.723452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.1:36317 (local address 127.13.109.3:38259)
0811 19:00:57.723569 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:57.723574 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:57.723938 (+ 364us) server_negotiation.cc:407] Connection header received
0811 19:00:57.724083 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:57.724086 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:57.724132 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:57.724224 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:00:57.724488 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:57.725037 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:57.726172 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:57.726574 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:57.726715 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:57.726775 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:57.727044 (+ 269us) server_negotiation.cc:299] Negotiation successful
0811 19:00:57.727118 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 19:00:57.727257 25382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:57.723414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.1:42107 (local address 127.13.109.2:46181)
0811 19:00:57.723569 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:57.723574 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:57.723826 (+ 252us) server_negotiation.cc:407] Connection header received
0811 19:00:57.723962 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:57.723966 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:57.724030 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:57.724106 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:00:57.724407 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:57.724966 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:57.726186 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:57.726571 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:57.726717 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:57.726776 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:57.727028 (+ 252us) server_negotiation.cc:299] Negotiation successful
0811 19:00:57.727103 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 19:00:57.727280 25392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:57.723307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.2:46181 (local address 127.13.109.1:42107)
0811 19:00:57.723716 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:57.723729 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:57.723839 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:57.724132 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:57.724135 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:57.724149 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:00:57.724293 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:57.724298 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:57.725076 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:57.725081 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:57.726050 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:57.726060 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:57.726717 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:57.726720 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:57.726880 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:57.726920 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 19:00:57.727009 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 19:00:57.727097 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"mutex_wait_us":30,"thread_start_us":115,"threads_started":1}
I20260811 19:00:57.727286 25393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:57.723307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.3:38259 (local address 127.13.109.1:36317)
0811 19:00:57.723833 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:57.723851 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:57.723976 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:57.724235 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:57.724238 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:57.724254 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:00:57.724379 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:57.724385 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:57.725144 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:57.725149 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:57.726044 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:57.726054 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:57.726715 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:57.726718 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:57.726915 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:57.726934 (+ 19us) client_negotiation.cc:769] Sending connection context
0811 19:00:57.727015 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 19:00:57.727092 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":437,"mutex_wait_us":105,"thread_start_us":108,"threads_started":1}
I20260811 19:00:57.728159 25163 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77a9013d2d9e44c982d238de3d469850" candidate_uuid: "e49407102e8d42ea98655358ce020b88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "222dca4ec73d42448a7ae1520cb16f8f" is_pre_election: true
I20260811 19:00:57.728157 25313 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77a9013d2d9e44c982d238de3d469850" candidate_uuid: "e49407102e8d42ea98655358ce020b88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05898439786a4076b4704825521128bc" is_pre_election: true
I20260811 19:00:57.728467 25313 raft_consensus.cc:2454] T 77a9013d2d9e44c982d238de3d469850 P 05898439786a4076b4704825521128bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e49407102e8d42ea98655358ce020b88 in term 0.
I20260811 19:00:57.728473 25163 raft_consensus.cc:2454] T 77a9013d2d9e44c982d238de3d469850 P 222dca4ec73d42448a7ae1520cb16f8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e49407102e8d42ea98655358ce020b88 in term 0.
I20260811 19:00:57.728964 24949 leader_election.cc:302] T 77a9013d2d9e44c982d238de3d469850 P e49407102e8d42ea98655358ce020b88 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 05898439786a4076b4704825521128bc, 222dca4ec73d42448a7ae1520cb16f8f, e49407102e8d42ea98655358ce020b88; no voters:
I20260811 19:00:57.729267 25389 raft_consensus.cc:2781] T 77a9013d2d9e44c982d238de3d469850 P e49407102e8d42ea98655358ce020b88 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 19:00:57.729346 25389 raft_consensus.cc:482] T 77a9013d2d9e44c982d238de3d469850 P e49407102e8d42ea98655358ce020b88 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 19:00:57.729394 25389 raft_consensus.cc:3037] T 77a9013d2d9e44c982d238de3d469850 P e49407102e8d42ea98655358ce020b88 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:00:57.730545 25389 raft_consensus.cc:504] T 77a9013d2d9e44c982d238de3d469850 P e49407102e8d42ea98655358ce020b88 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e49407102e8d42ea98655358ce020b88" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 34779 } } peers { permanent_uuid: "05898439786a4076b4704825521128bc" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 38259 } } peers { permanent_uuid: "222dca4ec73d42448a7ae1520cb16f8f" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 46181 } }
I20260811 19:00:57.730969 25389 leader_election.cc:288] T 77a9013d2d9e44c982d238de3d469850 P e49407102e8d42ea98655358ce020b88 [CANDIDATE]: Term 1 election: Requested vote from peers 05898439786a4076b4704825521128bc (127.13.109.3:38259), 222dca4ec73d42448a7ae1520cb16f8f (127.13.109.2:46181)
I20260811 19:00:57.731436 25313 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77a9013d2d9e44c982d238de3d469850" candidate_uuid: "e49407102e8d42ea98655358ce020b88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05898439786a4076b4704825521128bc"
I20260811 19:00:57.731455 25163 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77a9013d2d9e44c982d238de3d469850" candidate_uuid: "e49407102e8d42ea98655358ce020b88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "222dca4ec73d42448a7ae1520cb16f8f"
I20260811 19:00:57.731590 25313 raft_consensus.cc:3037] T 77a9013d2d9e44c982d238de3d469850 P 05898439786a4076b4704825521128bc [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:00:57.731602 25163 raft_consensus.cc:3037] T 77a9013d2d9e44c982d238de3d469850 P 222dca4ec73d42448a7ae1520cb16f8f [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:00:57.732690 25163 raft_consensus.cc:2454] T 77a9013d2d9e44c982d238de3d469850 P 222dca4ec73d42448a7ae1520cb16f8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e49407102e8d42ea98655358ce020b88 in term 1.
I20260811 19:00:57.732733 25313 raft_consensus.cc:2454] T 77a9013d2d9e44c982d238de3d469850 P 05898439786a4076b4704825521128bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e49407102e8d42ea98655358ce020b88 in term 1.
I20260811 19:00:57.733031 24950 leader_election.cc:302] T 77a9013d2d9e44c982d238de3d469850 P e49407102e8d42ea98655358ce020b88 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 222dca4ec73d42448a7ae1520cb16f8f, e49407102e8d42ea98655358ce020b88; no voters:
I20260811 19:00:57.733235 25389 raft_consensus.cc:2781] T 77a9013d2d9e44c982d238de3d469850 P e49407102e8d42ea98655358ce020b88 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:00:57.733487 25389 raft_consensus.cc:686] T 77a9013d2d9e44c982d238de3d469850 P e49407102e8d42ea98655358ce020b88 [term 1 LEADER]: Becoming Leader. State: Replica: e49407102e8d42ea98655358ce020b88, State: Running, Role: LEADER
I20260811 19:00:57.733906 25389 consensus_queue.cc:237] T 77a9013d2d9e44c982d238de3d469850 P e49407102e8d42ea98655358ce020b88 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e49407102e8d42ea98655358ce020b88" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 34779 } } peers { permanent_uuid: "05898439786a4076b4704825521128bc" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 38259 } } peers { permanent_uuid: "222dca4ec73d42448a7ae1520cb16f8f" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 46181 } }
I20260811 19:00:57.737481 24860 catalog_manager.cc:5495] T 77a9013d2d9e44c982d238de3d469850 P e49407102e8d42ea98655358ce020b88 reported cstate change: term changed from 0 to 1, leader changed from <none> to e49407102e8d42ea98655358ce020b88 (127.13.109.1). New cstate: current_term: 1 leader_uuid: "e49407102e8d42ea98655358ce020b88" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e49407102e8d42ea98655358ce020b88" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 34779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05898439786a4076b4704825521128bc" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 38259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "222dca4ec73d42448a7ae1520cb16f8f" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 46181 } health_report { overall_health: UNKNOWN } } }
I20260811 19:00:57.761807 25379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:57.757677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46124 (local address 127.13.109.1:34779)
0811 19:00:57.757853 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:57.757858 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:57.757873 (+ 15us) server_negotiation.cc:407] Connection header received
0811 19:00:57.758040 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:57.758043 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:57.758121 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:57.758235 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:00:57.758482 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:57.758992 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:57.759871 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:57.760127 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:57.760483 (+ 356us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:57.760835 (+ 352us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:00:57.761304 (+ 469us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:00:57.761466 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:57.761551 (+ 85us) server_negotiation.cc:299] Negotiation successful
0811 19:00:57.761615 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 19:00:57.768023 25163 raft_consensus.cc:1261] T 77a9013d2d9e44c982d238de3d469850 P 222dca4ec73d42448a7ae1520cb16f8f [term 1 FOLLOWER]: Refusing update from remote peer e49407102e8d42ea98655358ce020b88: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:00:57.768023 25313 raft_consensus.cc:1261] T 77a9013d2d9e44c982d238de3d469850 P 05898439786a4076b4704825521128bc [term 1 FOLLOWER]: Refusing update from remote peer e49407102e8d42ea98655358ce020b88: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:00:57.768761 25389 consensus_queue.cc:1035] T 77a9013d2d9e44c982d238de3d469850 P e49407102e8d42ea98655358ce020b88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05898439786a4076b4704825521128bc" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 38259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:00:57.768968 25394 consensus_queue.cc:1035] T 77a9013d2d9e44c982d238de3d469850 P e49407102e8d42ea98655358ce020b88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "222dca4ec73d42448a7ae1520cb16f8f" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 46181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:00:57.776782 25400 mvcc.cc:205] Tried to move back new op lower bound from 7317401017410584576 to 7317401017281069056. Current Snapshot: MvccSnapshot[applied={T|T < 7317401017410584576}]
I20260811 19:00:57.778533 25399 mvcc.cc:205] Tried to move back new op lower bound from 7317401017410584576 to 7317401017281069056. Current Snapshot: MvccSnapshot[applied={T|T < 7317401017410584576}]
I20260811 19:00:57.787386 24862 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55292:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:00:57.787568 24862 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55292:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:00:57.790227 24862 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4dd67887d86a4ff3b4581124b628a0bc: Sending DeleteTablet for 3 replicas of tablet 77a9013d2d9e44c982d238de3d469850
I20260811 19:00:57.791103 24998 tablet_service.cc:1514] Processing DeleteTablet for tablet 77a9013d2d9e44c982d238de3d469850 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:00:57 UTC) from {username='kudu', principal='kudu/127.13.109.62@KRBTEST.COM'} at 127.0.0.1:46120
I20260811 19:00:57.791142 25290 tablet_service.cc:1514] Processing DeleteTablet for tablet 77a9013d2d9e44c982d238de3d469850 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:00:57 UTC) from {username='kudu', principal='kudu/127.13.109.62@KRBTEST.COM'} at 127.0.0.1:46164
I20260811 19:00:57.791249 25143 tablet_service.cc:1514] Processing DeleteTablet for tablet 77a9013d2d9e44c982d238de3d469850 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:00:57 UTC) from {username='kudu', principal='kudu/127.13.109.62@KRBTEST.COM'} at 127.0.0.1:48734
I20260811 19:00:57.791718 25409 tablet_replica.cc:330] T 77a9013d2d9e44c982d238de3d469850 P 05898439786a4076b4704825521128bc: stopping tablet replica
I20260811 19:00:57.791859 25410 tablet_replica.cc:330] T 77a9013d2d9e44c982d238de3d469850 P e49407102e8d42ea98655358ce020b88: stopping tablet replica
I20260811 19:00:57.792069 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 24929
I20260811 19:00:57.792182 25409 raft_consensus.cc:2229] T 77a9013d2d9e44c982d238de3d469850 P 05898439786a4076b4704825521128bc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:00:57.792241 25410 raft_consensus.cc:2229] T 77a9013d2d9e44c982d238de3d469850 P e49407102e8d42ea98655358ce020b88 [term 1 LEADER]: Raft consensus shutting down.
I20260811 19:00:57.792555 25409 raft_consensus.cc:2258] T 77a9013d2d9e44c982d238de3d469850 P 05898439786a4076b4704825521128bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:00:57.792820 25411 tablet_replica.cc:330] T 77a9013d2d9e44c982d238de3d469850 P 222dca4ec73d42448a7ae1520cb16f8f: stopping tablet replica
I20260811 19:00:57.793130 25411 raft_consensus.cc:2229] T 77a9013d2d9e44c982d238de3d469850 P 222dca4ec73d42448a7ae1520cb16f8f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:00:57.793395 25411 raft_consensus.cc:2258] T 77a9013d2d9e44c982d238de3d469850 P 222dca4ec73d42448a7ae1520cb16f8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:00:57.795229 25411 ts_tablet_manager.cc:1905] T 77a9013d2d9e44c982d238de3d469850 P 222dca4ec73d42448a7ae1520cb16f8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:00:57.795871 25409 ts_tablet_manager.cc:1905] T 77a9013d2d9e44c982d238de3d469850 P 05898439786a4076b4704825521128bc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:00:57.798236 25411 ts_tablet_manager.cc:1918] T 77a9013d2d9e44c982d238de3d469850 P 222dca4ec73d42448a7ae1520cb16f8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:00:57.798357 25411 log.cc:1196] T 77a9013d2d9e44c982d238de3d469850 P 222dca4ec73d42448a7ae1520cb16f8f: Deleting WAL directory at /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/minicluster-data/ts-1/wal/wals/77a9013d2d9e44c982d238de3d469850
I20260811 19:00:57.798686 25409 ts_tablet_manager.cc:1918] T 77a9013d2d9e44c982d238de3d469850 P 05898439786a4076b4704825521128bc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:00:57.798717 25411 ts_tablet_manager.cc:1939] T 77a9013d2d9e44c982d238de3d469850 P 222dca4ec73d42448a7ae1520cb16f8f: Deleting consensus metadata
I20260811 19:00:57.798790 25409 log.cc:1196] T 77a9013d2d9e44c982d238de3d469850 P 05898439786a4076b4704825521128bc: Deleting WAL directory at /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474792496492-13748-0/minicluster-data/ts-2/wal/wals/77a9013d2d9e44c982d238de3d469850
I20260811 19:00:57.799113 25409 ts_tablet_manager.cc:1939] T 77a9013d2d9e44c982d238de3d469850 P 05898439786a4076b4704825521128bc: Deleting consensus metadata
I20260811 19:00:57.800225 24847 catalog_manager.cc:4841] TS 05898439786a4076b4704825521128bc (127.13.109.3:38259): tablet 77a9013d2d9e44c982d238de3d469850 (table test-table [id=6d54c8ba3d3b4821afefad12ad690a09]) successfully deleted
I20260811 19:00:57.800282 24848 catalog_manager.cc:4841] TS 222dca4ec73d42448a7ae1520cb16f8f (127.13.109.2:46181): tablet 77a9013d2d9e44c982d238de3d469850 (table test-table [id=6d54c8ba3d3b4821afefad12ad690a09]) successfully deleted
W20260811 19:00:57.801501 24845 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.109.1:34779 (error 108)
I20260811 19:00:57.802280 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 25071
I20260811 19:00:57.803722 25381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:57.802563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.1:34779 (local address 127.0.0.1:46138)
0811 19:00:57.803499 (+ 936us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:57.803606 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.1:34779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":813}
W20260811 19:00:57.804682 24845 catalog_manager.cc:4584] TS e49407102e8d42ea98655358ce020b88 (127.13.109.1:34779): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 77a9013d2d9e44c982d238de3d469850: Network error: Client connection negotiation failed: client connection to 127.13.109.1:34779: connect: Connection refused (error 111)
I20260811 19:00:57.811484 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 25219
I20260811 19:00:57.820174 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 24827
2026-08-11T19:00:57Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2642 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:00:57 dist-test-slave-0kls krb5kdc[25415](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:00:57 dist-test-slave-0kls krb5kdc[25415](info): set up 2 sockets
Aug 11 19:00:57 dist-test-slave-0kls krb5kdc[25415](info): commencing operation
krb5kdc: starting...
W20260811 19:00:58.919873 13748 mini_kdc.cc:121] Time spent starting KDC: real 1.075s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/krb5kdc/test-user.keytab.
Aug 11 19:00:59 dist-test-slave-0kls krb5kdc[25415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474859, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:00:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:00:59Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.13.109.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.13.109.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.13.109.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/krb5kdc/rangeradmin_127.13.109.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.13.109.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/krb5kdc/rangeradmin_127.13.109.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.13.109.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.13.109.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.13.109.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/krb5kdc/rangerlookup_127.13.109.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.13.109.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/krb5kdc/rangerlookup_127.13.109.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.13.109.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.13.109.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.13.109.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/krb5kdc/HTTP_127.13.109.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.13.109.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/krb5kdc/HTTP_127.13.109.20@KRBTEST.COM.keytab.
I20260811 19:00:59.108282 13748 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskrGEMHD/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/minicluster-data/postgres -l logfile start
2026-08-11 19:01:01.795 UTC [25449] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-11 19:01:01.795 UTC [25449] LOG: listening on IPv4 address "127.13.109.20", port 35695
2026-08-11 19:01:01.799 UTC [25449] LOG: listening on Unix socket "/tmp/.s.PGSQL.35695"
2026-08-11 19:01:01.823 UTC [25453] LOG: database system was shut down at 2026-08-11 19:01:00 UTC
2026-08-11 19:01:01.827 UTC [25449] LOG: database system is ready to accept connections
I20260811 19:01:02.834196 13748 mini_postgres.cc:96] Postgres bound to 35695
2026-08-11 19:01:02.842 UTC [25462] FATAL: database "slave" does not exist
127.13.109.20:35695 - accepting connections
I20260811 19:01:02.842929 13748 mini_ranger.cc:161] Starting Ranger...
I20260811 19:01:02.860146 13748 mini_ranger.cc:84] Created miniranger Postgres user
I20260811 19:01:03.382361 13748 mini_ranger.cc:87] Created ranger Postgres database
I20260811 19:01:03.382475 13748 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/minicluster-data/ranger-admin
2026-08-11 19:01:03,730 [I] DB FLAVOR :POSTGRES
2026-08-11 19:01:03,730 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 19:01:03,731 [I] Checking connection..
2026-08-11 19:01:03,731 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrGEMHD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrGEMHD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.109.20:35695/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-11 19:01:04,015 [I] Checking connection passed.
2026-08-11 19:01:04,015 [I] --------- Verifying version history table ---------
2026-08-11 19:01:04,015 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrGEMHD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrGEMHD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.109.20:35695/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 19:01:04,270 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 19:01:04,270 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrGEMHD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrGEMHD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.109.20:35695/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 19:01:04,539 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 19:01:04,539 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-11 19:01:04,539 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrGEMHD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrGEMHD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.109.20:35695/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskrGEMHD/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-11 19:01:04.806 UTC [25529] WARNING: there is no transaction in progress
2026-08-11 19:01:04,814 [I] create_dbversion_catalog.sql file imported successfully
2026-08-11 19:01:04,815 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrGEMHD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrGEMHD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.109.20:35695/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 19:01:05,106 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-11 19:01:05,106 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 19:01:05,106 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrGEMHD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrGEMHD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.109.20:35695/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-11 19:01:05,381 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrGEMHD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrGEMHD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.109.20:35695/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-11 19:01:05,629 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrGEMHD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrGEMHD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.109.20:35695/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-0kls.c.gcp-upstream.internal','N') ;"
2026-08-11 19:01:05,872 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrGEMHD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrGEMHD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.109.20:35695/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 19:01:06,136 [I] Table x_portal_user does not exist in database ranger
2026-08-11 19:01:06,137 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrGEMHD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrGEMHD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.109.20:35695/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 19:01:06,393 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-11 19:01:06,393 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrGEMHD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrGEMHD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.109.20:35695/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 19:01:06,649 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrGEMHD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrGEMHD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.109.20:35695/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 19:01:06,908 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-11 19:01:06,908 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrGEMHD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrGEMHD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.109.20:35695/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskrGEMHD/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-11 19:01:07.974 UTC [25648] WARNING: there is no transaction in progress
2026-08-11 19:01:07.990 UTC [25648] WARNING: there is no transaction in progress
2026-08-11 19:01:08.010 UTC [25648] WARNING: there is no transaction in progress
2026-08-11 19:01:08.028 UTC [25648] WARNING: there is no transaction in progress
2026-08-11 19:01:08.046 UTC [25648] WARNING: there is no transaction in progress
2026-08-11 19:01:08.066 UTC [25648] WARNING: there is no transaction in progress
2026-08-11 19:01:08.130 UTC [25648] WARNING: there is no transaction in progress
2026-08-11 19:01:08.139 UTC [25648] WARNING: there is no transaction in progress
2026-08-11 19:01:08.155 UTC [25648] WARNING: there is no transaction in progress
2026-08-11 19:01:08.165 UTC [25648] WARNING: there is no transaction in progress
2026-08-11 19:01:08.176 UTC [25648] WARNING: there is no transaction in progress
2026-08-11 19:01:08.191 UTC [25648] WARNING: there is no transaction in progress
2026-08-11 19:01:08.201 UTC [25648] WARNING: there is no transaction in progress
2026-08-11 19:01:08.208 UTC [25648] WARNING: there is no transaction in progress
2026-08-11 19:01:08.217 UTC [25648] WARNING: there is no transaction in progress
2026-08-11 19:01:08.224 UTC [25648] WARNING: there is no transaction in progress
2026-08-11 19:01:08,952 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-11 19:01:08,953 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrGEMHD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrGEMHD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.109.20:35695/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-11 19:01:09,208 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-11 19:01:09,208 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrGEMHD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrGEMHD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.109.20:35695/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 19:01:09,489 [I] Table x_portal_user already exists in database 'ranger'
2026-08-11 19:01:09,489 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrGEMHD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrGEMHD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.109.20:35695/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 19:01:09,748 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-11 19:01:09,748 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrGEMHD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrGEMHD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.109.20:35695/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 19:01:10,151 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrGEMHD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrGEMHD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.109.20:35695/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 19:01:10,597 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrGEMHD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrGEMHD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.109.20:35695/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-0kls.c.gcp-upstream.internal';"
2026-08-11 19:01:11,032 [I] CORE_DB_SCHEMA import status has been updated
2026-08-11 19:01:11,033 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrGEMHD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrGEMHD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.109.20:35695/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-11 19:01:11,462 [I] DB_PATCHES have already been applied
I20260811 19:01:11.468888 13748 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/minicluster-data/ranger-admin:/tmp/dist-test-taskrGEMHD/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskrGEMHD/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskrGEMHD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrGEMHD/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260811 19:01:11.468993 13748 mini_ranger.cc:193] Using host: 127.13.109.20
I20260811 19:01:11.473867 13748 mini_ranger.cc:227] Ranger admin URL: http://127.13.109.20:34329
Aug 11, 2026 7:01:11 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 7:01:11 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 7:01:11 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/minicluster-data/ranger-admin/ews/webapp
Aug 11, 2026 7:01:11 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 11, 2026 7:01:11 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 11, 2026 7:01:12 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/minicluster-data/ranger-admin/ews/webapp].
Aug 11, 2026 7:01:12 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260811 19:01:12.672037 13748 mini_ranger.cc:160] Time spent starting Ranger: real 9.829s user 0.005s sys 0.009s
Aug 11, 2026 7:01:34 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 11, 2026 7:01:34 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 11, 2026 7:01:34 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 11, 2026 7:01:34 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 11, 2026 7:01:34 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 11, 2026 7:01:34 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 11, 2026 7:01:34 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 11, 2026 7:01:34 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 11, 2026 7:01:34 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 11, 2026 7:01:34 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 11, 2026 7:01:34 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 11, 2026 7:01:34 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 11, 2026 7:01:34 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 11, 2026 7:01:34 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 11, 2026 7:01:34 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 11, 2026 7:01:34 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 11, 2026 7:01:34 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 11, 2026 7:01:35 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260811 19:01:35.992348 13748 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.13.109.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.13.109.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.13.109.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/krb5kdc/rangerkms_127.13.109.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.13.109.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/krb5kdc/rangerkms_127.13.109.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.13.109.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.13.109.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.13.109.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/krb5kdc/HTTP_127.13.109.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.13.109.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/krb5kdc/HTTP_127.13.109.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 11 19:01:36 dist-test-slave-0kls krb5kdc[25415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474896, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260811 19:01:36.097967 13748 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260811 19:01:36.109947 13748 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260811 19:01:37.065727 13748 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260811 19:01:37.065887 13748 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/minicluster-data/ranger-kms
I20260811 19:01:37.066012 13748 mini_ranger_kms.cc:227] Using postgres at 127.13.109.20:35695
2026-08-11 19:01:37,332 [I] DB FLAVOR :POSTGRES
2026-08-11 19:01:37,332 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 19:01:37,332 [I] Checking connection
2026-08-11 19:01:37,332 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrGEMHD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.109.20:35695/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-11 19:01:37,611 [I] connection success
2026-08-11 19:01:37,611 [I] --------- Verifying Ranger DB tables ---------
2026-08-11 19:01:37,611 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-11 19:01:37,611 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrGEMHD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.109.20:35695/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-11 19:01:37,901 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-11 19:01:37,901 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 19:01:37,902 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-11 19:01:37,902 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrGEMHD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.109.20:35695/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-11 19:01:38,184 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260811 19:01:38.709019 13748 mini_ranger_kms.cc:310] Created kms service
I20260811 19:01:38.902175 13748 mini_ranger_kms.cc:326] Created kudu user
I20260811 19:01:38.954290 13748 mini_ranger_kms.cc:343] Created rangerkms user
Aug 11 19:01:38 dist-test-slave-0kls krb5kdc[25415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474896, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.13.109.20@KRBTEST.COM
I20260811 19:01:39.388829 13748 mini_ranger_kms.cc:384] Added ranger policy
I20260811 19:01:39.389001 13748 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/minicluster-data/ranger-kms:/tmp/dist-test-taskrGEMHD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrGEMHD/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskrGEMHD/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260811 19:01:39.389027 13748 mini_ranger_kms.cc:240] Using host: 127.13.109.20
I20260811 19:01:39.394774 13748 mini_ranger_kms.cc:276] Ranger KMS PID: 25844
I20260811 19:01:39.394944 13748 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.13.109.20:55937
Aug 11, 2026 7:01:40 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 7:01:40 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 7:01:40 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 11, 2026 7:01:40 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 11, 2026 7:01:41 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/minicluster-data/ranger-kms/ews/webapp].
Aug 11, 2026 7:01:41 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260811 19:01:41.799328 13748 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.701s user 0.029s sys 0.221s
I20260811 19:01:41.799486 13748 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260811 19:01:41.799623 13748 mini_ranger_kms.cc:401] 127.13.109.20:55937/kms/v1/keys
Aug 11 19:01:41 dist-test-slave-0kls krb5kdc[25415](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786474901, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.13.109.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 11 19:01:42 dist-test-slave-0kls krb5kdc[25415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474901, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.13.109.20@KRBTEST.COM for HTTP/127.13.109.20@KRBTEST.COM
Aug 11 19:01:42 dist-test-slave-0kls krb5kdc[25415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474901, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.13.109.20@KRBTEST.COM for HTTP/127.13.109.20@KRBTEST.COM
Aug 11 19:01:45 dist-test-slave-0kls krb5kdc[25415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474905, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.13.109.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.109.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.109.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab.
Entry for principal kudu/127.13.109.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab.
I20260811 19:01:45.237677 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.62:39523
--webserver_interface=127.13.109.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab
--principal=kudu/127.13.109.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:35779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.109.62:39523
--ranger_config_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.109.20:55937/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:01:45.386541 25899 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:01:45.386845 25899 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:01:45.386886 25899 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:01:45.391228 25899 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:01:45.391331 25899 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:01:45.391352 25899 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:01:45.391369 25899 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:01:45.391423 25899 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:01:45.397630 25899 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:35779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/minicluster-data/master-0/wal
--ranger_kms_url=127.13.109.20:55937/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.13.109.62:39523
--ranger_config_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab
--principal=kudu/127.13.109.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.62:39523
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.109.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25899
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:01:45.399008 25899 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:01:45.400197 25899 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:01:45.410169 25905 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:01:45.410136 25907 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:01:45.410132 25904 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:01:45.411058 25899 server_base.cc:1034] running on GCE node
I20260811 19:01:45.411737 25899 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:01:45.413048 25899 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:01:45.414237 25899 hybrid_clock.cc:648] HybridClock initialized: now 1786474905414215 us; error 41 us; skew 500 ppm
Aug 11 19:01:45 dist-test-slave-0kls krb5kdc[25415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474905, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:01:45.417834 25899 init.cc:377] Logged in from keytab as kudu/127.13.109.62@KRBTEST.COM (short username kudu)
I20260811 19:01:45.418876 25899 webserver.cc:460] Webserver started at http://127.13.109.62:39085/ using document root <none> and password file <none>
I20260811 19:01:45.419503 25899 fs_manager.cc:359] Metadata directory not provided
I20260811 19:01:45.419590 25899 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:01:45.419870 25899 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 19:01:45 dist-test-slave-0kls krb5kdc[25415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474905, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.62@KRBTEST.COM for HTTP/127.13.109.20@KRBTEST.COM
I20260811 19:01:45.472819 25899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/minicluster-data/master-0/data/instance:
uuid: "875233b2e7b6443e87a6df9563713dc7"
format_stamp: "Formatted at 2026-08-11 19:01:45 on dist-test-slave-0kls"
server_key: "b7c0bcf576fa872311fa9b0a151622ba"
server_key_iv: "5189545e09ba45b42f950c3a7f362833"
server_key_version: "kuduclusterkey@0"
I20260811 19:01:45.473546 25899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/minicluster-data/master-0/wal/instance:
uuid: "875233b2e7b6443e87a6df9563713dc7"
format_stamp: "Formatted at 2026-08-11 19:01:45 on dist-test-slave-0kls"
server_key: "b7c0bcf576fa872311fa9b0a151622ba"
server_key_iv: "5189545e09ba45b42f950c3a7f362833"
server_key_version: "kuduclusterkey@0"
I20260811 19:01:45.478072 25899 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 19:01:45.493382 25914 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:01:45.494959 25899 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 19:01:45.495090 25899 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/minicluster-data/master-0/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/minicluster-data/master-0/wal
uuid: "875233b2e7b6443e87a6df9563713dc7"
format_stamp: "Formatted at 2026-08-11 19:01:45 on dist-test-slave-0kls"
server_key: "b7c0bcf576fa872311fa9b0a151622ba"
server_key_iv: "5189545e09ba45b42f950c3a7f362833"
server_key_version: "kuduclusterkey@0"
I20260811 19:01:45.495273 25899 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:01:45.512511 25899 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:01:45.515964 25899 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:01:45.516264 25899 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:01:45.525676 25899 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.62:39523
I20260811 19:01:45.525799 25972 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.62:39523 every 8 connection(s)
I20260811 19:01:45.526829 25899 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
I20260811 19:01:45.528604 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 25899
I20260811 19:01:45.528746 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/minicluster-data/master-0/wal/instance
I20260811 19:01:45.529088 13748 external_mini_cluster.cc:1428] Setting key 9dea96df5cd0ad093bd0b1203f3c0890
I20260811 19:01:45.530371 25973 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:01:45.538026 25973 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 875233b2e7b6443e87a6df9563713dc7: Bootstrap starting.
Aug 11 19:01:45 dist-test-slave-0kls krb5kdc[25415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474905, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
I20260811 19:01:45.540977 25973 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 875233b2e7b6443e87a6df9563713dc7: Neither blocks nor log segments found. Creating new log.
I20260811 19:01:45.542492 25973 log.cc:824] T 00000000000000000000000000000000 P 875233b2e7b6443e87a6df9563713dc7: Log is configured to *not* fsync() on all Append() calls
I20260811 19:01:45.545167 25973 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 875233b2e7b6443e87a6df9563713dc7: No bootstrap required, opened a new log
I20260811 19:01:45.546566 25976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:45.530678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55218 (local address 127.13.109.62:39523)
0811 19:01:45.531195 (+ 517us) server_negotiation.cc:206] Beginning negotiation
0811 19:01:45.531215 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 19:01:45.531261 (+ 46us) server_negotiation.cc:407] Connection header received
0811 19:01:45.532140 (+ 879us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:01:45.532176 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:01:45.532611 (+ 435us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:01:45.533022 (+ 411us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:01:45.533965 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:45.535514 (+ 1549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:45.536763 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:45.537480 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:45.537724 (+ 244us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:45.540938 (+ 3214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:01:45.540946 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:01:45.540967 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:01:45.541007 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:01:45.543228 (+ 2221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:01:45.543803 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:01:45.543809 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:01:45.543814 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:01:45.543930 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:01:45.544253 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:01:45.544256 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:01:45.544258 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:01:45.544742 (+ 484us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:01:45.544975 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0811 19:01:45.545363 (+ 388us) server_negotiation.cc:299] Negotiation successful
0811 19:01:45.545716 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":153,"threads_started":1}
I20260811 19:01:45.549459 25973 raft_consensus.cc:348] T 00000000000000000000000000000000 P 875233b2e7b6443e87a6df9563713dc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "875233b2e7b6443e87a6df9563713dc7" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 39523 } }
I20260811 19:01:45.549777 25973 raft_consensus.cc:374] T 00000000000000000000000000000000 P 875233b2e7b6443e87a6df9563713dc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:01:45.549885 25973 raft_consensus.cc:729] T 00000000000000000000000000000000 P 875233b2e7b6443e87a6df9563713dc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 875233b2e7b6443e87a6df9563713dc7, State: Initialized, Role: FOLLOWER
I20260811 19:01:45.550371 25973 consensus_queue.cc:260] T 00000000000000000000000000000000 P 875233b2e7b6443e87a6df9563713dc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "875233b2e7b6443e87a6df9563713dc7" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 39523 } }
I20260811 19:01:45.550535 25973 raft_consensus.cc:388] T 00000000000000000000000000000000 P 875233b2e7b6443e87a6df9563713dc7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:01:45.550601 25973 raft_consensus.cc:482] T 00000000000000000000000000000000 P 875233b2e7b6443e87a6df9563713dc7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:01:45.550710 25973 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 875233b2e7b6443e87a6df9563713dc7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:01:45.552131 25973 raft_consensus.cc:504] T 00000000000000000000000000000000 P 875233b2e7b6443e87a6df9563713dc7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "875233b2e7b6443e87a6df9563713dc7" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 39523 } }
I20260811 19:01:45.552556 25973 leader_election.cc:302] T 00000000000000000000000000000000 P 875233b2e7b6443e87a6df9563713dc7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 875233b2e7b6443e87a6df9563713dc7; no voters:
I20260811 19:01:45.552968 25973 leader_election.cc:288] T 00000000000000000000000000000000 P 875233b2e7b6443e87a6df9563713dc7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:01:45.553432 25978 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 875233b2e7b6443e87a6df9563713dc7 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:01:45.554050 25978 raft_consensus.cc:686] T 00000000000000000000000000000000 P 875233b2e7b6443e87a6df9563713dc7 [term 1 LEADER]: Becoming Leader. State: Replica: 875233b2e7b6443e87a6df9563713dc7, State: Running, Role: LEADER
I20260811 19:01:45.554695 25978 consensus_queue.cc:237] T 00000000000000000000000000000000 P 875233b2e7b6443e87a6df9563713dc7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "875233b2e7b6443e87a6df9563713dc7" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 39523 } }
I20260811 19:01:45.554913 25973 sys_catalog.cc:564] T 00000000000000000000000000000000 P 875233b2e7b6443e87a6df9563713dc7 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:01:45.556694 25980 sys_catalog.cc:455] T 00000000000000000000000000000000 P 875233b2e7b6443e87a6df9563713dc7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 875233b2e7b6443e87a6df9563713dc7. Latest consensus state: current_term: 1 leader_uuid: "875233b2e7b6443e87a6df9563713dc7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "875233b2e7b6443e87a6df9563713dc7" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 39523 } } }
I20260811 19:01:45.556840 25980 sys_catalog.cc:458] T 00000000000000000000000000000000 P 875233b2e7b6443e87a6df9563713dc7 [sys.catalog]: This master's current role is: LEADER
I20260811 19:01:45.557184 25979 sys_catalog.cc:455] T 00000000000000000000000000000000 P 875233b2e7b6443e87a6df9563713dc7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "875233b2e7b6443e87a6df9563713dc7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "875233b2e7b6443e87a6df9563713dc7" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 39523 } } }
I20260811 19:01:45.557274 25979 sys_catalog.cc:458] T 00000000000000000000000000000000 P 875233b2e7b6443e87a6df9563713dc7 [sys.catalog]: This master's current role is: LEADER
I20260811 19:01:45.557488 25973 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 19:01:45.557751 25993 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:01:45.562033 25993 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:01:45.576611 25993 catalog_manager.cc:1347] Generated new cluster ID: 9bbfe09001c44c02b353f777cfae0543
I20260811 19:01:45.576737 25993 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:01:45.599489 25993 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:01:45.602855 25993 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:01:45.610937 25993 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 875233b2e7b6443e87a6df9563713dc7: Generated new TSK 0
I20260811 19:01:45.612174 25993 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.109.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.109.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.109.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.1.keytab.
Entry for principal kudu/127.13.109.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.1.keytab.
I20260811 19:01:46.393605 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.109.20:55937/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.1:0
--local_ip_for_outbound_sockets=127.13.109.1
--webserver_interface=127.13.109.1
--webserver_port=0
--tserver_master_addrs=127.13.109.62:39523
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.1.keytab
--principal=kudu/127.13.109.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:35779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:01:46.572770 26021 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:01:46.573122 26021 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:01:46.573232 26021 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:01:46.578856 26021 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:01:46.578953 26021 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:01:46.579044 26021 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.1
I20260811 19:01:46.587106 26021 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:35779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.13.109.20:55937/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.1.keytab
--principal=kudu/127.13.109.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.109.1
--webserver_port=0
--tserver_master_addrs=127.13.109.62:39523
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26021
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.1
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:01:46.588546 26021 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:01:46.589730 26021 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:01:46.602254 26028 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:01:46.602921 26021 server_base.cc:1034] running on GCE node
W20260811 19:01:46.605994 26031 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:01:46.606009 26029 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:01:46.606772 26021 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:01:46.607837 26021 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:01:46.609099 26021 hybrid_clock.cc:648] HybridClock initialized: now 1786474906609045 us; error 69 us; skew 500 ppm
Aug 11 19:01:46 dist-test-slave-0kls krb5kdc[25415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474906, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:01:46.613718 26021 init.cc:377] Logged in from keytab as kudu/127.13.109.1@KRBTEST.COM (short username kudu)
I20260811 19:01:46.615193 26021 webserver.cc:460] Webserver started at http://127.13.109.1:46863/ using document root <none> and password file <none>
I20260811 19:01:46.615970 26021 fs_manager.cc:359] Metadata directory not provided
I20260811 19:01:46.616050 26021 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:01:46.616255 26021 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 19:01:46 dist-test-slave-0kls krb5kdc[25415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474906, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.1@KRBTEST.COM for HTTP/127.13.109.20@KRBTEST.COM
I20260811 19:01:46.638033 26021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/minicluster-data/ts-0/data/instance:
uuid: "ff5aa0dda50842d284fab9bed12c6b95"
format_stamp: "Formatted at 2026-08-11 19:01:46 on dist-test-slave-0kls"
server_key: "8c7932ad6d6bef722451d200ff067eb9"
server_key_iv: "e6f5db2e807c90ce9db25a1988bce344"
server_key_version: "kuduclusterkey@0"
I20260811 19:01:46.638970 26021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/minicluster-data/ts-0/wal/instance:
uuid: "ff5aa0dda50842d284fab9bed12c6b95"
format_stamp: "Formatted at 2026-08-11 19:01:46 on dist-test-slave-0kls"
server_key: "8c7932ad6d6bef722451d200ff067eb9"
server_key_iv: "e6f5db2e807c90ce9db25a1988bce344"
server_key_version: "kuduclusterkey@0"
I20260811 19:01:46.644670 26021 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 19:01:46.683621 26039 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:01:46.685597 26021 fs_manager.cc:725] Time spent opening block manager: real 0.012s user 0.004s sys 0.000s
I20260811 19:01:46.685802 26021 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/minicluster-data/ts-0/wal
uuid: "ff5aa0dda50842d284fab9bed12c6b95"
format_stamp: "Formatted at 2026-08-11 19:01:46 on dist-test-slave-0kls"
server_key: "8c7932ad6d6bef722451d200ff067eb9"
server_key_iv: "e6f5db2e807c90ce9db25a1988bce344"
server_key_version: "kuduclusterkey@0"
I20260811 19:01:46.685976 26021 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:01:46.713773 26021 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:01:46.717764 26021 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:01:46.718065 26021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:01:46.718842 26021 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:01:46.720116 26021 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:01:46.720242 26021 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:01:46.720369 26021 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:01:46.720443 26021 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:01:46.737056 26021 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.1:45959
I20260811 19:01:46.738464 26021 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
I20260811 19:01:46.747972 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 26021
I20260811 19:01:46.748108 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/minicluster-data/ts-0/wal/instance
I20260811 19:01:46.748445 13748 external_mini_cluster.cc:1428] Setting key a65318874741c5580e7bf82ad52c5493
I20260811 19:01:46.749091 26166 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.1:45959 every 8 connection(s)
WARNING: no policy specified for kudu/127.13.109.2@KRBTEST.COM; defaulting to no policy
Aug 11 19:01:46 dist-test-slave-0kls krb5kdc[25415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474906, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.1@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
I20260811 19:01:46.781935 26170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:46.752129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:39523 (local address 127.13.109.1:47815)
0811 19:01:46.758282 (+ 6153us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:46.758372 (+ 90us) client_negotiation.cc:174] Beginning negotiation
0811 19:01:46.759685 (+ 1313us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:01:46.767173 (+ 7488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:01:46.767188 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:01:46.767628 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:01:46.768446 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:01:46.768472 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:01:46.769819 (+ 1347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:01:46.769823 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:01:46.771720 (+ 1897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:01:46.771737 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:01:46.772620 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:01:46.772626 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:01:46.772879 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:46.773629 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:01:46.773665 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:01:46.775476 (+ 1811us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:01:46.778003 (+ 2527us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:01:46.778011 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:01:46.778034 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:01:46.778440 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:01:46.778814 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:01:46.778817 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:01:46.778820 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:01:46.779033 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:01:46.779597 (+ 564us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:01:46.779605 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:01:46.780009 (+ 404us) client_negotiation.cc:769] Sending connection context
0811 19:01:46.780245 (+ 236us) client_negotiation.cc:240] Negotiation successful
0811 19:01:46.780651 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5836,"thread_start_us":151,"threads_started":1}
I20260811 19:01:46.782922 26171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:46.753330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.1:47815 (local address 127.13.109.62:39523)
0811 19:01:46.766076 (+ 12746us) server_negotiation.cc:206] Beginning negotiation
0811 19:01:46.766081 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:01:46.766104 (+ 23us) server_negotiation.cc:407] Connection header received
0811 19:01:46.766166 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:01:46.766169 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:01:46.766235 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:01:46.766366 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:01:46.768724 (+ 2358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:46.769595 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:46.771907 (+ 2312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:46.772273 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:46.772388 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:46.775753 (+ 3365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:01:46.775757 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:01:46.775761 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:01:46.775793 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:01:46.777828 (+ 2035us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:01:46.778578 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:01:46.778582 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:01:46.778583 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:01:46.778700 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:01:46.779154 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:01:46.779157 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:01:46.779160 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:01:46.779434 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:01:46.779539 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 19:01:46.782550 (+ 3011us) server_negotiation.cc:299] Negotiation successful
0811 19:01:46.782740 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12622,"thread_start_us":124,"threads_started":1}
I20260811 19:01:46.784960 26167 heartbeater.cc:347] Connected to a master server at 127.13.109.62:39523
I20260811 19:01:46.785295 26167 heartbeater.cc:464] Registering TS with master...
I20260811 19:01:46.786108 26167 heartbeater.cc:511] Master 127.13.109.62:39523 requested a full tablet report, sending...
I20260811 19:01:46.788595 25932 ts_manager.cc:194] Registered new tserver with Master: ff5aa0dda50842d284fab9bed12c6b95 (127.13.109.1:45959)
I20260811 19:01:46.790377 25932 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.109.1@KRBTEST.COM'} at 127.13.109.1:47815
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.109.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.109.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.2.keytab.
Entry for principal kudu/127.13.109.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.2.keytab.
I20260811 19:01:46.817535 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.109.20:55937/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.2:0
--local_ip_for_outbound_sockets=127.13.109.2
--webserver_interface=127.13.109.2
--webserver_port=0
--tserver_master_addrs=127.13.109.62:39523
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.2.keytab
--principal=kudu/127.13.109.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:35779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 19:01:46 dist-test-slave-0kls krb5kdc[25415](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786474906, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260811 19:01:47.002952 26175 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:01:47.003444 26175 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:01:47.003544 26175 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:01:47.010346 26175 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:01:47.010488 26175 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:01:47.010598 26175 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.2
I20260811 19:01:47.018781 26175 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:35779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.13.109.20:55937/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.2.keytab
--principal=kudu/127.13.109.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.109.2
--webserver_port=0
--tserver_master_addrs=127.13.109.62:39523
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26175
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.2
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:01:47.020496 26175 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:01:47.022378 26175 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:01:47.038725 26184 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:01:47.039695 26187 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:01:47.042296 26185 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:01:47.046051 26175 server_base.cc:1034] running on GCE node
I20260811 19:01:47.046712 26175 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:01:47.047605 26175 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:01:47.048813 26175 hybrid_clock.cc:648] HybridClock initialized: now 1786474907048777 us; error 55 us; skew 500 ppm
Aug 11 19:01:47 dist-test-slave-0kls krb5kdc[25415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474907, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:01:47.053841 26175 init.cc:377] Logged in from keytab as kudu/127.13.109.2@KRBTEST.COM (short username kudu)
I20260811 19:01:47.055178 26175 webserver.cc:460] Webserver started at http://127.13.109.2:34577/ using document root <none> and password file <none>
I20260811 19:01:47.056038 26175 fs_manager.cc:359] Metadata directory not provided
I20260811 19:01:47.056208 26175 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:01:47.056548 26175 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 19:01:47 dist-test-slave-0kls krb5kdc[25415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474907, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.2@KRBTEST.COM for HTTP/127.13.109.20@KRBTEST.COM
I20260811 19:01:47.074364 26175 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/minicluster-data/ts-1/data/instance:
uuid: "6e9687a46fe4401a80a979cf130e22c5"
format_stamp: "Formatted at 2026-08-11 19:01:47 on dist-test-slave-0kls"
server_key: "23230321323cb8b549645b1d5a019ff5"
server_key_iv: "fed9ec3a9500e20a106ceb5afee3b7a2"
server_key_version: "kuduclusterkey@0"
I20260811 19:01:47.075233 26175 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/minicluster-data/ts-1/wal/instance:
uuid: "6e9687a46fe4401a80a979cf130e22c5"
format_stamp: "Formatted at 2026-08-11 19:01:47 on dist-test-slave-0kls"
server_key: "23230321323cb8b549645b1d5a019ff5"
server_key_iv: "fed9ec3a9500e20a106ceb5afee3b7a2"
server_key_version: "kuduclusterkey@0"
I20260811 19:01:47.080791 26175 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 19:01:47.106346 26196 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:01:47.108299 26175 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.002s sys 0.000s
I20260811 19:01:47.108549 26175 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/minicluster-data/ts-1/wal
uuid: "6e9687a46fe4401a80a979cf130e22c5"
format_stamp: "Formatted at 2026-08-11 19:01:47 on dist-test-slave-0kls"
server_key: "23230321323cb8b549645b1d5a019ff5"
server_key_iv: "fed9ec3a9500e20a106ceb5afee3b7a2"
server_key_version: "kuduclusterkey@0"
I20260811 19:01:47.108821 26175 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:01:47.132352 26175 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:01:47.136268 26175 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:01:47.136511 26175 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:01:47.137244 26175 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:01:47.138345 26175 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:01:47.138420 26175 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:01:47.138491 26175 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:01:47.138525 26175 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:01:47.150884 26175 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.2:34667
I20260811 19:01:47.151189 26335 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.2:34667 every 8 connection(s)
I20260811 19:01:47.152009 26175 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
I20260811 19:01:47.160943 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 26175
I20260811 19:01:47.161109 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/minicluster-data/ts-1/wal/instance
I20260811 19:01:47.161496 13748 external_mini_cluster.cc:1428] Setting key 0909290b1816929f634e7137702bb5df
Aug 11 19:01:47 dist-test-slave-0kls krb5kdc[25415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474907, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.2@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
I20260811 19:01:47.170293 26171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:47.154727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.2:54225 (local address 127.13.109.62:39523)
0811 19:01:47.154906 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 19:01:47.154911 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:01:47.156133 (+ 1222us) server_negotiation.cc:407] Connection header received
0811 19:01:47.157311 (+ 1178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:01:47.157315 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:01:47.157373 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:01:47.157482 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:01:47.158898 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:47.159696 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:47.161331 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:47.161689 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:47.161817 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:47.165333 (+ 3516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:01:47.165337 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:01:47.165340 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:01:47.165371 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:01:47.167600 (+ 2229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:01:47.168253 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:01:47.168259 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:01:47.168264 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:01:47.168376 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:01:47.168804 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:01:47.168809 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:01:47.168811 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:01:47.169124 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:01:47.169241 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 19:01:47.169902 (+ 661us) server_negotiation.cc:299] Negotiation successful
0811 19:01:47.170067 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 19:01:47.171144 26338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:47.155261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:39523 (local address 127.13.109.2:54225)
0811 19:01:47.155919 (+ 658us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:47.155978 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 19:01:47.157063 (+ 1085us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:01:47.157693 (+ 630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:01:47.157700 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:01:47.158145 (+ 445us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:01:47.158741 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:01:47.158757 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:01:47.159899 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:01:47.159902 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:01:47.161164 (+ 1262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:01:47.161180 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:01:47.161924 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:01:47.161929 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:01:47.162176 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:47.162924 (+ 748us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:01:47.162958 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:01:47.165119 (+ 2161us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:01:47.167752 (+ 2633us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:01:47.167760 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:01:47.167780 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:01:47.168119 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:01:47.168491 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:01:47.168495 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:01:47.168497 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:01:47.168679 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:01:47.169251 (+ 572us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:01:47.169262 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:01:47.169573 (+ 311us) client_negotiation.cc:769] Sending connection context
0811 19:01:47.169800 (+ 227us) client_negotiation.cc:240] Negotiation successful
0811 19:01:47.170198 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":192,"threads_started":1}
I20260811 19:01:47.172605 26336 heartbeater.cc:347] Connected to a master server at 127.13.109.62:39523
I20260811 19:01:47.172902 26336 heartbeater.cc:464] Registering TS with master...
I20260811 19:01:47.173489 26336 heartbeater.cc:511] Master 127.13.109.62:39523 requested a full tablet report, sending...
I20260811 19:01:47.174823 25932 ts_manager.cc:194] Registered new tserver with Master: 6e9687a46fe4401a80a979cf130e22c5 (127.13.109.2:34667)
I20260811 19:01:47.175755 25932 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.109.2@KRBTEST.COM'} at 127.13.109.2:54225
WARNING: no policy specified for kudu/127.13.109.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.109.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.109.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.3.keytab.
Entry for principal kudu/127.13.109.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.3.keytab.
I20260811 19:01:47.220886 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.109.20:55937/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.3:0
--local_ip_for_outbound_sockets=127.13.109.3
--webserver_interface=127.13.109.3
--webserver_port=0
--tserver_master_addrs=127.13.109.62:39523
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.3.keytab
--principal=kudu/127.13.109.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:35779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 19:01:47 dist-test-slave-0kls krb5kdc[25415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474906, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.62@KRBTEST.COM for HTTP/127.13.109.20@KRBTEST.COM
W20260811 19:01:47.406451 26342 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:01:47.407136 26342 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:01:47.407256 26342 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:01:47.413394 26342 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:01:47.413491 26342 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:01:47.413578 26342 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.3
I20260811 19:01:47.422284 26342 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:35779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.13.109.20:55937/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.3.keytab
--principal=kudu/127.13.109.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.109.3
--webserver_port=0
--tserver_master_addrs=127.13.109.62:39523
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26342
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.3
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:01:47.424213 26342 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:01:47.425774 26342 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:01:47.439620 26352 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:01:47.442075 26342 server_base.cc:1034] running on GCE node
W20260811 19:01:47.446636 26350 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:01:47.447152 26348 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:01:47.448671 26342 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:01:47.449625 26342 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:01:47.457077 26342 hybrid_clock.cc:648] HybridClock initialized: now 1786474907456987 us; error 108 us; skew 500 ppm
Aug 11 19:01:47 dist-test-slave-0kls krb5kdc[25415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474907, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:01:47.461225 26342 init.cc:377] Logged in from keytab as kudu/127.13.109.3@KRBTEST.COM (short username kudu)
I20260811 19:01:47.462540 26342 webserver.cc:460] Webserver started at http://127.13.109.3:42177/ using document root <none> and password file <none>
I20260811 19:01:47.463337 26342 fs_manager.cc:359] Metadata directory not provided
I20260811 19:01:47.463475 26342 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:01:47.463770 26342 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 19:01:47 dist-test-slave-0kls krb5kdc[25415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474907, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.3@KRBTEST.COM for HTTP/127.13.109.20@KRBTEST.COM
I20260811 19:01:47.487249 26342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/minicluster-data/ts-2/data/instance:
uuid: "eac5e1f0f3f94e1cbc7e2ede7e5b35ca"
format_stamp: "Formatted at 2026-08-11 19:01:47 on dist-test-slave-0kls"
server_key: "43c4f6b746ae31758ae7835189f5ca09"
server_key_iv: "1371a7347d7a196c9377c71db006a6c3"
server_key_version: "kuduclusterkey@0"
I20260811 19:01:47.488756 26342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/minicluster-data/ts-2/wal/instance:
uuid: "eac5e1f0f3f94e1cbc7e2ede7e5b35ca"
format_stamp: "Formatted at 2026-08-11 19:01:47 on dist-test-slave-0kls"
server_key: "43c4f6b746ae31758ae7835189f5ca09"
server_key_iv: "1371a7347d7a196c9377c71db006a6c3"
server_key_version: "kuduclusterkey@0"
I20260811 19:01:47.499784 26342 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.000s sys 0.004s
I20260811 19:01:47.530864 26359 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:01:47.535460 26342 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.003s sys 0.001s
I20260811 19:01:47.535647 26342 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/minicluster-data/ts-2/wal
uuid: "eac5e1f0f3f94e1cbc7e2ede7e5b35ca"
format_stamp: "Formatted at 2026-08-11 19:01:47 on dist-test-slave-0kls"
server_key: "43c4f6b746ae31758ae7835189f5ca09"
server_key_iv: "1371a7347d7a196c9377c71db006a6c3"
server_key_version: "kuduclusterkey@0"
I20260811 19:01:47.535815 26342 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:01:47.560168 26342 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:01:47.563546 26342 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:01:47.563776 26342 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:01:47.564498 26342 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:01:47.565481 26342 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:01:47.565553 26342 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:01:47.565659 26342 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:01:47.565717 26342 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:01:47.578251 26342 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.3:40205
I20260811 19:01:47.579449 26342 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
I20260811 19:01:47.580876 26488 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.3:40205 every 8 connection(s)
I20260811 19:01:47.589532 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 26342
I20260811 19:01:47.589679 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/minicluster-data/ts-2/wal/instance
I20260811 19:01:47.590164 13748 external_mini_cluster.cc:1428] Setting key 69eedc9d6c841b5fa0cda97ba3dfe023
Aug 11 19:01:47 dist-test-slave-0kls krb5kdc[25415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474907, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.3@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
I20260811 19:01:47.598889 26171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:47.582044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.3:57133 (local address 127.13.109.62:39523)
0811 19:01:47.582205 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0811 19:01:47.582210 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:01:47.584097 (+ 1887us) server_negotiation.cc:407] Connection header received
0811 19:01:47.585338 (+ 1241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:01:47.585342 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:01:47.585412 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:01:47.585529 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:01:47.587156 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:47.588018 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:47.589875 (+ 1857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:47.590263 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:47.590364 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:47.593976 (+ 3612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:01:47.593980 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:01:47.593983 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:01:47.594017 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:01:47.595953 (+ 1936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:01:47.596721 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:01:47.596725 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:01:47.596727 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:01:47.596825 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:01:47.597335 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:01:47.597339 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:01:47.597341 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:01:47.597594 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:01:47.597714 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 19:01:47.598561 (+ 847us) server_negotiation.cc:299] Negotiation successful
0811 19:01:47.598716 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 19:01:47.599982 26491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:47.582571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:39523 (local address 127.13.109.3:57133)
0811 19:01:47.583872 (+ 1301us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:47.583938 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0811 19:01:47.585055 (+ 1117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:01:47.585809 (+ 754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:01:47.585820 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:01:47.586227 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:01:47.586950 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:01:47.586970 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:01:47.588205 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:01:47.588209 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:01:47.589666 (+ 1457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:01:47.589680 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:01:47.590921 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:01:47.590925 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:01:47.591157 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:47.591918 (+ 761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:01:47.591941 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:01:47.593713 (+ 1772us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:01:47.596106 (+ 2393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:01:47.596113 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:01:47.596135 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:01:47.596574 (+ 439us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:01:47.596941 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:01:47.596944 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:01:47.596946 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:01:47.597176 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:01:47.597732 (+ 556us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:01:47.597741 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:01:47.598171 (+ 430us) client_negotiation.cc:769] Sending connection context
0811 19:01:47.598441 (+ 270us) client_negotiation.cc:240] Negotiation successful
0811 19:01:47.598838 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1063,"thread_start_us":147,"threads_started":1}
I20260811 19:01:47.601432 26489 heartbeater.cc:347] Connected to a master server at 127.13.109.62:39523
I20260811 19:01:47.601730 26489 heartbeater.cc:464] Registering TS with master...
I20260811 19:01:47.602527 26489 heartbeater.cc:511] Master 127.13.109.62:39523 requested a full tablet report, sending...
I20260811 19:01:47.603847 25932 ts_manager.cc:194] Registered new tserver with Master: eac5e1f0f3f94e1cbc7e2ede7e5b35ca (127.13.109.3:40205)
I20260811 19:01:47.604660 25932 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.109.3@KRBTEST.COM'} at 127.13.109.3:57133
I20260811 19:01:47.604882 13748 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 19:01:47.617904 26171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:47.607274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55228 (local address 127.13.109.62:39523)
0811 19:01:47.607397 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0811 19:01:47.607401 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:01:47.608019 (+ 618us) server_negotiation.cc:407] Connection header received
0811 19:01:47.608208 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:01:47.608211 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:01:47.608259 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:01:47.608310 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:01:47.609548 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:47.610409 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:47.611603 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:47.611959 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:47.612075 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:47.613980 (+ 1905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:01:47.613984 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:01:47.613987 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:01:47.614021 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:01:47.615859 (+ 1838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:01:47.616476 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:01:47.616480 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:01:47.616482 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:01:47.616577 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:01:47.616905 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:01:47.616908 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:01:47.616910 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:01:47.617144 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:01:47.617281 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 19:01:47.617562 (+ 281us) server_negotiation.cc:299] Negotiation successful
0811 19:01:47.617708 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 19:01:47.621747 25932 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55228:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 19:01:47.626740 25932 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 19:01:47.647989 26503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:47.639778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.3:40205 (local address 127.0.0.1:53684)
0811 19:01:47.641993 (+ 2215us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:47.642008 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 19:01:47.642106 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:01:47.642809 (+ 703us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:01:47.642815 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:01:47.642838 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:01:47.642976 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:01:47.642982 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:01:47.644418 (+ 1436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:01:47.644422 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:01:47.645969 (+ 1547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:01:47.645981 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:01:47.647378 (+ 1397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:01:47.647382 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:01:47.647509 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:47.647551 (+ 42us) client_negotiation.cc:769] Sending connection context
0811 19:01:47.647688 (+ 137us) client_negotiation.cc:240] Negotiation successful
0811 19:01:47.647829 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2144,"thread_start_us":109,"threads_started":1}
I20260811 19:01:47.648741 26508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:47.640679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53684 (local address 127.13.109.3:40205)
0811 19:01:47.641889 (+ 1210us) server_negotiation.cc:206] Beginning negotiation
0811 19:01:47.641902 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 19:01:47.642162 (+ 260us) server_negotiation.cc:407] Connection header received
0811 19:01:47.642246 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:01:47.642251 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:01:47.642470 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:01:47.642620 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:01:47.643105 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:47.644288 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:47.646149 (+ 1861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:47.647021 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:47.647129 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:47.647308 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0811 19:01:47.648445 (+ 1137us) server_negotiation.cc:299] Negotiation successful
0811 19:01:47.648571 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1078,"thread_start_us":111,"threads_started":1}
I20260811 19:01:47.650096 26505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:47.639982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.2:34667 (local address 127.0.0.1:32944)
0811 19:01:47.642315 (+ 2333us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:47.642327 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 19:01:47.642473 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:01:47.643258 (+ 785us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:01:47.643261 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:01:47.643278 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:01:47.643388 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:01:47.643395 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:01:47.645661 (+ 2266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:01:47.645664 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:01:47.647731 (+ 2067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:01:47.647742 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:01:47.649643 (+ 1901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:01:47.649647 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:01:47.649744 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:47.649770 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 19:01:47.649874 (+ 104us) client_negotiation.cc:240] Negotiation successful
0811 19:01:47.649955 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2271,"thread_start_us":99,"threads_started":1}
I20260811 19:01:47.650648 26507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:47.640279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32944 (local address 127.13.109.2:34667)
0811 19:01:47.640635 (+ 356us) server_negotiation.cc:206] Beginning negotiation
0811 19:01:47.641494 (+ 859us) server_negotiation.cc:399] Waiting for connection header
0811 19:01:47.642409 (+ 915us) server_negotiation.cc:407] Connection header received
0811 19:01:47.642679 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:01:47.642685 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:01:47.642873 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:01:47.643037 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:01:47.644357 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:47.645507 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:47.647897 (+ 2390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:47.649278 (+ 1381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:47.649374 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:47.649532 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0811 19:01:47.650385 (+ 853us) server_negotiation.cc:299] Negotiation successful
0811 19:01:47.650504 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":236,"thread_start_us":119,"threads_started":1}
I20260811 19:01:47.652292 26410 tablet_service.cc:1467] Processing CreateTablet for tablet 680667f002b9450ab0b64130a17178b3 (DEFAULT_TABLE table=test-table [id=6400b8dcf39e4551963a7f0eb9c85571]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:01:47.653571 26410 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 680667f002b9450ab0b64130a17178b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:01:47.658631 26506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:47.637723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.1:45959 (local address 127.0.0.1:33710)
0811 19:01:47.641374 (+ 3651us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:47.641432 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 19:01:47.641561 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:01:47.652773 (+ 11212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:01:47.652777 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:01:47.652800 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:01:47.652989 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:01:47.652998 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:01:47.654733 (+ 1735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:01:47.654738 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:01:47.656964 (+ 2226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:01:47.656977 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:01:47.658165 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:01:47.658169 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:01:47.658274 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:47.658307 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 19:01:47.658368 (+ 61us) client_negotiation.cc:240] Negotiation successful
0811 19:01:47.658458 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3471,"thread_start_us":173,"threads_started":1}
I20260811 19:01:47.658567 26250 tablet_service.cc:1467] Processing CreateTablet for tablet 680667f002b9450ab0b64130a17178b3 (DEFAULT_TABLE table=test-table [id=6400b8dcf39e4551963a7f0eb9c85571]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:01:47.659795 26504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:47.637880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33710 (local address 127.13.109.1:45959)
0811 19:01:47.652223 (+ 14343us) server_negotiation.cc:206] Beginning negotiation
0811 19:01:47.652231 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:01:47.652260 (+ 29us) server_negotiation.cc:407] Connection header received
0811 19:01:47.652350 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:01:47.652356 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:01:47.652588 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:01:47.653173 (+ 585us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:01:47.653347 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:47.654578 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:47.657136 (+ 2558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:47.658016 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:47.659239 (+ 1223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:47.659409 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0811 19:01:47.659470 (+ 61us) server_negotiation.cc:299] Negotiation successful
0811 19:01:47.659619 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14196,"thread_start_us":98,"threads_started":1}
I20260811 19:01:47.659896 26250 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 680667f002b9450ab0b64130a17178b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:01:47.663233 26097 tablet_service.cc:1467] Processing CreateTablet for tablet 680667f002b9450ab0b64130a17178b3 (DEFAULT_TABLE table=test-table [id=6400b8dcf39e4551963a7f0eb9c85571]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:01:47.664439 26097 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 680667f002b9450ab0b64130a17178b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:01:47.675299 26510 tablet_bootstrap.cc:492] T 680667f002b9450ab0b64130a17178b3 P ff5aa0dda50842d284fab9bed12c6b95: Bootstrap starting.
I20260811 19:01:47.677039 26509 tablet_bootstrap.cc:492] T 680667f002b9450ab0b64130a17178b3 P eac5e1f0f3f94e1cbc7e2ede7e5b35ca: Bootstrap starting.
I20260811 19:01:47.678165 26510 tablet_bootstrap.cc:654] T 680667f002b9450ab0b64130a17178b3 P ff5aa0dda50842d284fab9bed12c6b95: Neither blocks nor log segments found. Creating new log.
I20260811 19:01:47.679302 26510 log.cc:824] T 680667f002b9450ab0b64130a17178b3 P ff5aa0dda50842d284fab9bed12c6b95: Log is configured to *not* fsync() on all Append() calls
I20260811 19:01:47.680110 26509 tablet_bootstrap.cc:654] T 680667f002b9450ab0b64130a17178b3 P eac5e1f0f3f94e1cbc7e2ede7e5b35ca: Neither blocks nor log segments found. Creating new log.
I20260811 19:01:47.680992 26509 log.cc:824] T 680667f002b9450ab0b64130a17178b3 P eac5e1f0f3f94e1cbc7e2ede7e5b35ca: Log is configured to *not* fsync() on all Append() calls
I20260811 19:01:47.681830 26511 tablet_bootstrap.cc:492] T 680667f002b9450ab0b64130a17178b3 P 6e9687a46fe4401a80a979cf130e22c5: Bootstrap starting.
I20260811 19:01:47.682268 26510 tablet_bootstrap.cc:492] T 680667f002b9450ab0b64130a17178b3 P ff5aa0dda50842d284fab9bed12c6b95: No bootstrap required, opened a new log
I20260811 19:01:47.682494 26510 ts_tablet_manager.cc:1397] T 680667f002b9450ab0b64130a17178b3 P ff5aa0dda50842d284fab9bed12c6b95: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260811 19:01:47.684232 26511 tablet_bootstrap.cc:654] T 680667f002b9450ab0b64130a17178b3 P 6e9687a46fe4401a80a979cf130e22c5: Neither blocks nor log segments found. Creating new log.
I20260811 19:01:47.685763 26511 log.cc:824] T 680667f002b9450ab0b64130a17178b3 P 6e9687a46fe4401a80a979cf130e22c5: Log is configured to *not* fsync() on all Append() calls
I20260811 19:01:47.688450 26509 tablet_bootstrap.cc:492] T 680667f002b9450ab0b64130a17178b3 P eac5e1f0f3f94e1cbc7e2ede7e5b35ca: No bootstrap required, opened a new log
I20260811 19:01:47.688716 26509 ts_tablet_manager.cc:1397] T 680667f002b9450ab0b64130a17178b3 P eac5e1f0f3f94e1cbc7e2ede7e5b35ca: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.000s
I20260811 19:01:47.689065 26510 raft_consensus.cc:348] T 680667f002b9450ab0b64130a17178b3 P ff5aa0dda50842d284fab9bed12c6b95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff5aa0dda50842d284fab9bed12c6b95" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 45959 } } peers { permanent_uuid: "eac5e1f0f3f94e1cbc7e2ede7e5b35ca" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 40205 } } peers { permanent_uuid: "6e9687a46fe4401a80a979cf130e22c5" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34667 } }
I20260811 19:01:47.689524 26510 raft_consensus.cc:374] T 680667f002b9450ab0b64130a17178b3 P ff5aa0dda50842d284fab9bed12c6b95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:01:47.689659 26510 raft_consensus.cc:729] T 680667f002b9450ab0b64130a17178b3 P ff5aa0dda50842d284fab9bed12c6b95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff5aa0dda50842d284fab9bed12c6b95, State: Initialized, Role: FOLLOWER
I20260811 19:01:47.690383 26510 consensus_queue.cc:260] T 680667f002b9450ab0b64130a17178b3 P ff5aa0dda50842d284fab9bed12c6b95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff5aa0dda50842d284fab9bed12c6b95" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 45959 } } peers { permanent_uuid: "eac5e1f0f3f94e1cbc7e2ede7e5b35ca" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 40205 } } peers { permanent_uuid: "6e9687a46fe4401a80a979cf130e22c5" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34667 } }
I20260811 19:01:47.692034 26510 ts_tablet_manager.cc:1428] T 680667f002b9450ab0b64130a17178b3 P ff5aa0dda50842d284fab9bed12c6b95: Time spent starting tablet: real 0.009s user 0.005s sys 0.003s
I20260811 19:01:47.692633 26167 heartbeater.cc:503] Master 127.13.109.62:39523 was elected leader, sending a full tablet report...
I20260811 19:01:47.694921 26509 raft_consensus.cc:348] T 680667f002b9450ab0b64130a17178b3 P eac5e1f0f3f94e1cbc7e2ede7e5b35ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff5aa0dda50842d284fab9bed12c6b95" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 45959 } } peers { permanent_uuid: "eac5e1f0f3f94e1cbc7e2ede7e5b35ca" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 40205 } } peers { permanent_uuid: "6e9687a46fe4401a80a979cf130e22c5" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34667 } }
I20260811 19:01:47.695478 26509 raft_consensus.cc:374] T 680667f002b9450ab0b64130a17178b3 P eac5e1f0f3f94e1cbc7e2ede7e5b35ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:01:47.695586 26509 raft_consensus.cc:729] T 680667f002b9450ab0b64130a17178b3 P eac5e1f0f3f94e1cbc7e2ede7e5b35ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac5e1f0f3f94e1cbc7e2ede7e5b35ca, State: Initialized, Role: FOLLOWER
I20260811 19:01:47.696251 26509 consensus_queue.cc:260] T 680667f002b9450ab0b64130a17178b3 P eac5e1f0f3f94e1cbc7e2ede7e5b35ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff5aa0dda50842d284fab9bed12c6b95" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 45959 } } peers { permanent_uuid: "eac5e1f0f3f94e1cbc7e2ede7e5b35ca" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 40205 } } peers { permanent_uuid: "6e9687a46fe4401a80a979cf130e22c5" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34667 } }
I20260811 19:01:47.699360 26509 ts_tablet_manager.cc:1428] T 680667f002b9450ab0b64130a17178b3 P eac5e1f0f3f94e1cbc7e2ede7e5b35ca: Time spent starting tablet: real 0.011s user 0.009s sys 0.000s
I20260811 19:01:47.699956 26489 heartbeater.cc:503] Master 127.13.109.62:39523 was elected leader, sending a full tablet report...
I20260811 19:01:47.701293 26511 tablet_bootstrap.cc:492] T 680667f002b9450ab0b64130a17178b3 P 6e9687a46fe4401a80a979cf130e22c5: No bootstrap required, opened a new log
I20260811 19:01:47.701589 26511 ts_tablet_manager.cc:1397] T 680667f002b9450ab0b64130a17178b3 P 6e9687a46fe4401a80a979cf130e22c5: Time spent bootstrapping tablet: real 0.020s user 0.003s sys 0.004s
I20260811 19:01:47.708220 26511 raft_consensus.cc:348] T 680667f002b9450ab0b64130a17178b3 P 6e9687a46fe4401a80a979cf130e22c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff5aa0dda50842d284fab9bed12c6b95" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 45959 } } peers { permanent_uuid: "eac5e1f0f3f94e1cbc7e2ede7e5b35ca" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 40205 } } peers { permanent_uuid: "6e9687a46fe4401a80a979cf130e22c5" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34667 } }
I20260811 19:01:47.708652 26511 raft_consensus.cc:374] T 680667f002b9450ab0b64130a17178b3 P 6e9687a46fe4401a80a979cf130e22c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:01:47.708745 26511 raft_consensus.cc:729] T 680667f002b9450ab0b64130a17178b3 P 6e9687a46fe4401a80a979cf130e22c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e9687a46fe4401a80a979cf130e22c5, State: Initialized, Role: FOLLOWER
I20260811 19:01:47.709401 26511 consensus_queue.cc:260] T 680667f002b9450ab0b64130a17178b3 P 6e9687a46fe4401a80a979cf130e22c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff5aa0dda50842d284fab9bed12c6b95" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 45959 } } peers { permanent_uuid: "eac5e1f0f3f94e1cbc7e2ede7e5b35ca" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 40205 } } peers { permanent_uuid: "6e9687a46fe4401a80a979cf130e22c5" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34667 } }
I20260811 19:01:47.710988 26336 heartbeater.cc:503] Master 127.13.109.62:39523 was elected leader, sending a full tablet report...
I20260811 19:01:47.710803 26511 ts_tablet_manager.cc:1428] T 680667f002b9450ab0b64130a17178b3 P 6e9687a46fe4401a80a979cf130e22c5: Time spent starting tablet: real 0.009s user 0.006s sys 0.001s
W20260811 19:01:47.750587 26168 tablet.cc:2369] T 680667f002b9450ab0b64130a17178b3 P ff5aa0dda50842d284fab9bed12c6b95: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 11 19:01:47 dist-test-slave-0kls krb5kdc[25415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474906, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.62@KRBTEST.COM for HTTP/127.13.109.20@KRBTEST.COM
I20260811 19:01:47.789284 26517 raft_consensus.cc:482] T 680667f002b9450ab0b64130a17178b3 P 6e9687a46fe4401a80a979cf130e22c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:01:47.789558 26517 raft_consensus.cc:504] T 680667f002b9450ab0b64130a17178b3 P 6e9687a46fe4401a80a979cf130e22c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff5aa0dda50842d284fab9bed12c6b95" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 45959 } } peers { permanent_uuid: "eac5e1f0f3f94e1cbc7e2ede7e5b35ca" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 40205 } } peers { permanent_uuid: "6e9687a46fe4401a80a979cf130e22c5" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34667 } }
I20260811 19:01:47.791184 26517 leader_election.cc:288] T 680667f002b9450ab0b64130a17178b3 P 6e9687a46fe4401a80a979cf130e22c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ff5aa0dda50842d284fab9bed12c6b95 (127.13.109.1:45959), eac5e1f0f3f94e1cbc7e2ede7e5b35ca (127.13.109.3:40205)
I20260811 19:01:47.796890 26504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:47.791899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.2:45729 (local address 127.13.109.1:45959)
0811 19:01:47.792057 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 19:01:47.792063 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:01:47.792417 (+ 354us) server_negotiation.cc:407] Connection header received
0811 19:01:47.792593 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:01:47.792597 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:01:47.792656 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:01:47.792742 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:01:47.793062 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:47.793887 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:47.795573 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:47.796233 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:47.796370 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:47.796427 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 19:01:47.796678 (+ 251us) server_negotiation.cc:299] Negotiation successful
0811 19:01:47.796759 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 19:01:47.796898 26522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:47.791414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.1:45959 (local address 127.13.109.2:45729)
0811 19:01:47.792328 (+ 914us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:47.792344 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 19:01:47.792468 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:01:47.792758 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:01:47.792761 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:01:47.792779 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:01:47.792912 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:01:47.792921 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:01:47.794037 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:01:47.794041 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:01:47.795408 (+ 1367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:01:47.795420 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:01:47.796388 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:01:47.796392 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:01:47.796504 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:47.796534 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 19:01:47.796621 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 19:01:47.796718 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":823,"thread_start_us":126,"threads_started":1}
I20260811 19:01:47.797897 26118 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "680667f002b9450ab0b64130a17178b3" candidate_uuid: "6e9687a46fe4401a80a979cf130e22c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff5aa0dda50842d284fab9bed12c6b95" is_pre_election: true
I20260811 19:01:47.798306 26118 raft_consensus.cc:2454] T 680667f002b9450ab0b64130a17178b3 P ff5aa0dda50842d284fab9bed12c6b95 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e9687a46fe4401a80a979cf130e22c5 in term 0.
I20260811 19:01:47.799256 26520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:47.791571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.3:40205 (local address 127.13.109.2:55875)
0811 19:01:47.794037 (+ 2466us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:47.794055 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 19:01:47.794268 (+ 213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:01:47.794713 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:01:47.794716 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:01:47.794734 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:01:47.794856 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:01:47.794861 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:01:47.795809 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:01:47.795813 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:01:47.796954 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:01:47.797474 (+ 520us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:01:47.798748 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:01:47.798752 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:01:47.798852 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:47.798878 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 19:01:47.798964 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 19:01:47.799073 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2390,"mutex_wait_us":18,"thread_start_us":114,"threads_started":1}
I20260811 19:01:47.799705 26508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:47.792010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.2:55875 (local address 127.13.109.3:40205)
0811 19:01:47.792139 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 19:01:47.792143 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:01:47.794140 (+ 1997us) server_negotiation.cc:407] Connection header received
0811 19:01:47.794425 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:01:47.794430 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:01:47.794494 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:01:47.794548 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:01:47.795001 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:47.795640 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:47.797594 (+ 1954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:47.798246 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:47.798376 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:47.798431 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 19:01:47.799466 (+ 1035us) server_negotiation.cc:299] Negotiation successful
0811 19:01:47.799567 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 19:01:47.799698 26197 leader_election.cc:302] T 680667f002b9450ab0b64130a17178b3 P 6e9687a46fe4401a80a979cf130e22c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e9687a46fe4401a80a979cf130e22c5, ff5aa0dda50842d284fab9bed12c6b95; no voters:
I20260811 19:01:47.800079 26517 raft_consensus.cc:2781] T 680667f002b9450ab0b64130a17178b3 P 6e9687a46fe4401a80a979cf130e22c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 19:01:47.800192 26517 raft_consensus.cc:482] T 680667f002b9450ab0b64130a17178b3 P 6e9687a46fe4401a80a979cf130e22c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 19:01:47.800246 26517 raft_consensus.cc:3037] T 680667f002b9450ab0b64130a17178b3 P 6e9687a46fe4401a80a979cf130e22c5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:01:47.800420 26421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "680667f002b9450ab0b64130a17178b3" candidate_uuid: "6e9687a46fe4401a80a979cf130e22c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac5e1f0f3f94e1cbc7e2ede7e5b35ca" is_pre_election: true
I20260811 19:01:47.800729 26421 raft_consensus.cc:2454] T 680667f002b9450ab0b64130a17178b3 P eac5e1f0f3f94e1cbc7e2ede7e5b35ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e9687a46fe4401a80a979cf130e22c5 in term 0.
I20260811 19:01:47.801836 26517 raft_consensus.cc:504] T 680667f002b9450ab0b64130a17178b3 P 6e9687a46fe4401a80a979cf130e22c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff5aa0dda50842d284fab9bed12c6b95" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 45959 } } peers { permanent_uuid: "eac5e1f0f3f94e1cbc7e2ede7e5b35ca" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 40205 } } peers { permanent_uuid: "6e9687a46fe4401a80a979cf130e22c5" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34667 } }
I20260811 19:01:47.802250 26517 leader_election.cc:288] T 680667f002b9450ab0b64130a17178b3 P 6e9687a46fe4401a80a979cf130e22c5 [CANDIDATE]: Term 1 election: Requested vote from peers ff5aa0dda50842d284fab9bed12c6b95 (127.13.109.1:45959), eac5e1f0f3f94e1cbc7e2ede7e5b35ca (127.13.109.3:40205)
I20260811 19:01:47.802829 26421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "680667f002b9450ab0b64130a17178b3" candidate_uuid: "6e9687a46fe4401a80a979cf130e22c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac5e1f0f3f94e1cbc7e2ede7e5b35ca"
I20260811 19:01:47.803020 26421 raft_consensus.cc:3037] T 680667f002b9450ab0b64130a17178b3 P eac5e1f0f3f94e1cbc7e2ede7e5b35ca [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:01:47.803094 26118 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "680667f002b9450ab0b64130a17178b3" candidate_uuid: "6e9687a46fe4401a80a979cf130e22c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff5aa0dda50842d284fab9bed12c6b95"
I20260811 19:01:47.803252 26118 raft_consensus.cc:3037] T 680667f002b9450ab0b64130a17178b3 P ff5aa0dda50842d284fab9bed12c6b95 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:01:47.804337 26421 raft_consensus.cc:2454] T 680667f002b9450ab0b64130a17178b3 P eac5e1f0f3f94e1cbc7e2ede7e5b35ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e9687a46fe4401a80a979cf130e22c5 in term 1.
I20260811 19:01:47.804577 26118 raft_consensus.cc:2454] T 680667f002b9450ab0b64130a17178b3 P ff5aa0dda50842d284fab9bed12c6b95 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e9687a46fe4401a80a979cf130e22c5 in term 1.
I20260811 19:01:47.805094 26199 leader_election.cc:302] T 680667f002b9450ab0b64130a17178b3 P 6e9687a46fe4401a80a979cf130e22c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e9687a46fe4401a80a979cf130e22c5, eac5e1f0f3f94e1cbc7e2ede7e5b35ca; no voters:
I20260811 19:01:47.805372 26517 raft_consensus.cc:2781] T 680667f002b9450ab0b64130a17178b3 P 6e9687a46fe4401a80a979cf130e22c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:01:47.805629 26517 raft_consensus.cc:686] T 680667f002b9450ab0b64130a17178b3 P 6e9687a46fe4401a80a979cf130e22c5 [term 1 LEADER]: Becoming Leader. State: Replica: 6e9687a46fe4401a80a979cf130e22c5, State: Running, Role: LEADER
I20260811 19:01:47.806066 26517 consensus_queue.cc:237] T 680667f002b9450ab0b64130a17178b3 P 6e9687a46fe4401a80a979cf130e22c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff5aa0dda50842d284fab9bed12c6b95" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 45959 } } peers { permanent_uuid: "eac5e1f0f3f94e1cbc7e2ede7e5b35ca" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 40205 } } peers { permanent_uuid: "6e9687a46fe4401a80a979cf130e22c5" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34667 } }
I20260811 19:01:47.809947 25932 catalog_manager.cc:5495] T 680667f002b9450ab0b64130a17178b3 P 6e9687a46fe4401a80a979cf130e22c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6e9687a46fe4401a80a979cf130e22c5 (127.13.109.2). New cstate: current_term: 1 leader_uuid: "6e9687a46fe4401a80a979cf130e22c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff5aa0dda50842d284fab9bed12c6b95" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 45959 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eac5e1f0f3f94e1cbc7e2ede7e5b35ca" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 40205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e9687a46fe4401a80a979cf130e22c5" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34667 } health_report { overall_health: HEALTHY } } }
W20260811 19:01:47.831853 26490 tablet.cc:2369] T 680667f002b9450ab0b64130a17178b3 P eac5e1f0f3f94e1cbc7e2ede7e5b35ca: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 19:01:47.854628 26507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:47.850450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32950 (local address 127.13.109.2:34667)
0811 19:01:47.850775 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0811 19:01:47.850780 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:01:47.850797 (+ 17us) server_negotiation.cc:407] Connection header received
0811 19:01:47.850909 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:01:47.850913 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:01:47.850986 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:01:47.851122 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:01:47.851412 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:47.851943 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:47.852874 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:47.853106 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:47.853240 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:47.853621 (+ 381us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:01:47.854203 (+ 582us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:01:47.854352 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0811 19:01:47.854411 (+ 59us) server_negotiation.cc:299] Negotiation successful
0811 19:01:47.854475 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":223}
I20260811 19:01:47.862298 26421 raft_consensus.cc:1261] T 680667f002b9450ab0b64130a17178b3 P eac5e1f0f3f94e1cbc7e2ede7e5b35ca [term 1 FOLLOWER]: Refusing update from remote peer 6e9687a46fe4401a80a979cf130e22c5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:01:47.862480 26118 raft_consensus.cc:1261] T 680667f002b9450ab0b64130a17178b3 P ff5aa0dda50842d284fab9bed12c6b95 [term 1 FOLLOWER]: Refusing update from remote peer 6e9687a46fe4401a80a979cf130e22c5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:01:47.863236 26517 consensus_queue.cc:1035] T 680667f002b9450ab0b64130a17178b3 P 6e9687a46fe4401a80a979cf130e22c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff5aa0dda50842d284fab9bed12c6b95" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 45959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:01:47.866489 26517 consensus_queue.cc:1035] T 680667f002b9450ab0b64130a17178b3 P 6e9687a46fe4401a80a979cf130e22c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac5e1f0f3f94e1cbc7e2ede7e5b35ca" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 40205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:01:47.877167 26527 mvcc.cc:205] Tried to move back new op lower bound from 7317401222594473984 to 7317401222377394176. Current Snapshot: MvccSnapshot[applied={T|T < 7317401222594473984}]
I20260811 19:01:47.889603 26528 mvcc.cc:205] Tried to move back new op lower bound from 7317401222594473984 to 7317401222377394176. Current Snapshot: MvccSnapshot[applied={T|T < 7317401222594473984}]
I20260811 19:01:47.889568 25932 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55228:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:01:47.889753 25932 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55228:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:01:47.892628 25932 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 875233b2e7b6443e87a6df9563713dc7: Sending DeleteTablet for 3 replicas of tablet 680667f002b9450ab0b64130a17178b3
I20260811 19:01:47.893464 26097 tablet_service.cc:1514] Processing DeleteTablet for tablet 680667f002b9450ab0b64130a17178b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:01:47 UTC) from {username='kudu', principal='kudu/127.13.109.62@KRBTEST.COM'} at 127.0.0.1:33710
I20260811 19:01:47.894726 26410 tablet_service.cc:1514] Processing DeleteTablet for tablet 680667f002b9450ab0b64130a17178b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:01:47 UTC) from {username='kudu', principal='kudu/127.13.109.62@KRBTEST.COM'} at 127.0.0.1:53684
I20260811 19:01:47.895908 26250 tablet_service.cc:1514] Processing DeleteTablet for tablet 680667f002b9450ab0b64130a17178b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:01:47 UTC) from {username='kudu', principal='kudu/127.13.109.62@KRBTEST.COM'} at 127.0.0.1:32944
I20260811 19:01:47.897681 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 26021
I20260811 19:01:47.913004 26540 tablet_replica.cc:330] T 680667f002b9450ab0b64130a17178b3 P 6e9687a46fe4401a80a979cf130e22c5: stopping tablet replica
I20260811 19:01:47.913563 26539 tablet_replica.cc:330] T 680667f002b9450ab0b64130a17178b3 P eac5e1f0f3f94e1cbc7e2ede7e5b35ca: stopping tablet replica
I20260811 19:01:47.914144 26540 raft_consensus.cc:2229] T 680667f002b9450ab0b64130a17178b3 P 6e9687a46fe4401a80a979cf130e22c5 [term 1 LEADER]: Raft consensus shutting down.
I20260811 19:01:47.914654 26539 raft_consensus.cc:2229] T 680667f002b9450ab0b64130a17178b3 P eac5e1f0f3f94e1cbc7e2ede7e5b35ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:01:47.915205 26540 raft_consensus.cc:2258] T 680667f002b9450ab0b64130a17178b3 P 6e9687a46fe4401a80a979cf130e22c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:01:47.915647 26539 raft_consensus.cc:2258] T 680667f002b9450ab0b64130a17178b3 P eac5e1f0f3f94e1cbc7e2ede7e5b35ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:01:47.923477 26539 ts_tablet_manager.cc:1905] T 680667f002b9450ab0b64130a17178b3 P eac5e1f0f3f94e1cbc7e2ede7e5b35ca: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 19:01:47.926874 25915 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.109.1:45959 (error 108)
I20260811 19:01:47.927353 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 26175
I20260811 19:01:47.927438 26539 ts_tablet_manager.cc:1918] T 680667f002b9450ab0b64130a17178b3 P eac5e1f0f3f94e1cbc7e2ede7e5b35ca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:01:47.927575 26539 log.cc:1196] T 680667f002b9450ab0b64130a17178b3 P eac5e1f0f3f94e1cbc7e2ede7e5b35ca: Deleting WAL directory at /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474792496492-13748-0/minicluster-data/ts-2/wal/wals/680667f002b9450ab0b64130a17178b3
I20260811 19:01:47.928728 26539 ts_tablet_manager.cc:1939] T 680667f002b9450ab0b64130a17178b3 P eac5e1f0f3f94e1cbc7e2ede7e5b35ca: Deleting consensus metadata
I20260811 19:01:47.930459 26506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:47.928069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.1:45959 (local address 127.0.0.1:33718)
0811 19:01:47.930261 (+ 2192us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:47.930352 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.1:45959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2063}
I20260811 19:01:47.930824 25917 catalog_manager.cc:4841] TS eac5e1f0f3f94e1cbc7e2ede7e5b35ca (127.13.109.3:40205): tablet 680667f002b9450ab0b64130a17178b3 (table test-table [id=6400b8dcf39e4551963a7f0eb9c85571]) successfully deleted
W20260811 19:01:47.934130 25915 catalog_manager.cc:4584] TS ff5aa0dda50842d284fab9bed12c6b95 (127.13.109.1:45959): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 680667f002b9450ab0b64130a17178b3: Network error: Client connection negotiation failed: client connection to 127.13.109.1:45959: connect: Connection refused (error 111)
W20260811 19:01:47.943665 26506 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:01:47.943200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.2:34667 (local address 127.0.0.1:32958)
0811 19:01:47.943443 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:47.943514 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.2:34667: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":137}
I20260811 19:01:47.944175 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 26342
I20260811 19:01:47.954957 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 25899
I20260811 19:01:47.966735 13748 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260811 19:01:48.307806 13748 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260811 19:01:48.307951 13748 mini_ranger.cc:66] Stopping Ranger...
I20260811 19:01:48.664371 13748 mini_ranger.cc:68] Stopped Ranger
2026-08-11 19:01:48.664 UTC [25449] LOG: received smart shutdown request
2026-08-11 19:01:48.670 UTC [25449] LOG: background worker "logical replication launcher" (PID 25460) exited with exit code 1
2026-08-11 19:01:48.670 UTC [25455] LOG: shutting down
2026-08-11 19:01:48.866 UTC [25449] LOG: database system is shut down
2026-08-11T19:01:48Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (51169 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:01:49 dist-test-slave-0kls krb5kdc[26552](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:01:49 dist-test-slave-0kls krb5kdc[26552](info): set up 2 sockets
Aug 11 19:01:49 dist-test-slave-0kls krb5kdc[26552](info): commencing operation
krb5kdc: starting...
W20260811 19:01:50.105129 13748 mini_kdc.cc:121] Time spent starting KDC: real 1.091s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/krb5kdc/test-user.keytab.
Aug 11 19:01:50 dist-test-slave-0kls krb5kdc[26552](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474910, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:01:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:01:50Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.13.109.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.13.109.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.13.109.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/krb5kdc/rangeradmin_127.13.109.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.13.109.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/krb5kdc/rangeradmin_127.13.109.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.13.109.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.13.109.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.13.109.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/krb5kdc/rangerlookup_127.13.109.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.13.109.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/krb5kdc/rangerlookup_127.13.109.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.13.109.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.13.109.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.13.109.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/krb5kdc/HTTP_127.13.109.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.13.109.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/krb5kdc/HTTP_127.13.109.20@KRBTEST.COM.keytab.
I20260811 19:01:50.306262 13748 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskrGEMHD/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/postgres -l logfile start
2026-08-11 19:01:53.032 UTC [26587] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-11 19:01:53.033 UTC [26587] LOG: listening on IPv4 address "127.13.109.20", port 35369
2026-08-11 19:01:53.037 UTC [26587] LOG: listening on Unix socket "/tmp/.s.PGSQL.35369"
2026-08-11 19:01:53.060 UTC [26591] LOG: database system was shut down at 2026-08-11 19:01:51 UTC
2026-08-11 19:01:53.064 UTC [26587] LOG: database system is ready to accept connections
I20260811 19:01:54.072468 13748 mini_postgres.cc:96] Postgres bound to 35369
2026-08-11 19:01:54.080 UTC [26599] FATAL: database "slave" does not exist
127.13.109.20:35369 - accepting connections
I20260811 19:01:54.081516 13748 mini_ranger.cc:161] Starting Ranger...
I20260811 19:01:54.099185 13748 mini_ranger.cc:84] Created miniranger Postgres user
I20260811 19:01:54.647790 13748 mini_ranger.cc:87] Created ranger Postgres database
I20260811 19:01:54.647888 13748 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ranger-admin
2026-08-11 19:01:54,956 [I] DB FLAVOR :POSTGRES
2026-08-11 19:01:54,957 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 19:01:54,957 [I] Checking connection..
2026-08-11 19:01:54,957 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrGEMHD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrGEMHD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.109.20:35369/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-11 19:01:55,228 [I] Checking connection passed.
2026-08-11 19:01:55,228 [I] --------- Verifying version history table ---------
2026-08-11 19:01:55,228 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrGEMHD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrGEMHD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.109.20:35369/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 19:01:55,509 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 19:01:55,510 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrGEMHD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrGEMHD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.109.20:35369/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 19:01:55,783 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 19:01:55,783 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-11 19:01:55,783 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrGEMHD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrGEMHD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.109.20:35369/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskrGEMHD/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-11 19:01:56.037 UTC [26677] WARNING: there is no transaction in progress
2026-08-11 19:01:56,045 [I] create_dbversion_catalog.sql file imported successfully
2026-08-11 19:01:56,046 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrGEMHD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrGEMHD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.109.20:35369/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 19:01:56,295 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-11 19:01:56,295 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 19:01:56,296 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrGEMHD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrGEMHD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.109.20:35369/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-11 19:01:56,542 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrGEMHD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrGEMHD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.109.20:35369/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-11 19:01:56,798 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrGEMHD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrGEMHD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.109.20:35369/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-0kls.c.gcp-upstream.internal','N') ;"
2026-08-11 19:01:57,061 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrGEMHD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrGEMHD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.109.20:35369/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 19:01:57,340 [I] Table x_portal_user does not exist in database ranger
2026-08-11 19:01:57,340 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrGEMHD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrGEMHD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.109.20:35369/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 19:01:57,603 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-11 19:01:57,603 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrGEMHD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrGEMHD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.109.20:35369/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 19:01:57,881 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrGEMHD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrGEMHD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.109.20:35369/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 19:01:58,156 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-11 19:01:58,156 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrGEMHD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrGEMHD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.109.20:35369/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskrGEMHD/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-11 19:01:59.203 UTC [26793] WARNING: there is no transaction in progress
2026-08-11 19:01:59.221 UTC [26793] WARNING: there is no transaction in progress
2026-08-11 19:01:59.240 UTC [26793] WARNING: there is no transaction in progress
2026-08-11 19:01:59.258 UTC [26793] WARNING: there is no transaction in progress
2026-08-11 19:01:59.275 UTC [26793] WARNING: there is no transaction in progress
2026-08-11 19:01:59.295 UTC [26793] WARNING: there is no transaction in progress
2026-08-11 19:01:59.359 UTC [26793] WARNING: there is no transaction in progress
2026-08-11 19:01:59.367 UTC [26793] WARNING: there is no transaction in progress
2026-08-11 19:01:59.383 UTC [26793] WARNING: there is no transaction in progress
2026-08-11 19:01:59.395 UTC [26793] WARNING: there is no transaction in progress
2026-08-11 19:01:59.408 UTC [26793] WARNING: there is no transaction in progress
2026-08-11 19:01:59.424 UTC [26793] WARNING: there is no transaction in progress
2026-08-11 19:01:59.435 UTC [26793] WARNING: there is no transaction in progress
2026-08-11 19:01:59.443 UTC [26793] WARNING: there is no transaction in progress
2026-08-11 19:01:59.450 UTC [26793] WARNING: there is no transaction in progress
2026-08-11 19:01:59.458 UTC [26793] WARNING: there is no transaction in progress
2026-08-11 19:02:00,202 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-11 19:02:00,202 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrGEMHD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrGEMHD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.109.20:35369/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-11 19:02:00,450 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-11 19:02:00,450 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrGEMHD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrGEMHD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.109.20:35369/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 19:02:00,715 [I] Table x_portal_user already exists in database 'ranger'
2026-08-11 19:02:00,716 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrGEMHD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrGEMHD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.109.20:35369/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 19:02:00,996 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-11 19:02:00,996 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrGEMHD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrGEMHD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.109.20:35369/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 19:02:01,261 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrGEMHD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrGEMHD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.109.20:35369/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 19:02:01,525 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrGEMHD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrGEMHD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.109.20:35369/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-0kls.c.gcp-upstream.internal';"
2026-08-11 19:02:01,778 [I] CORE_DB_SCHEMA import status has been updated
2026-08-11 19:02:01,779 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrGEMHD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrGEMHD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.109.20:35369/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-11 19:02:02,054 [I] DB_PATCHES have already been applied
I20260811 19:02:02.060897 13748 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ranger-admin:/tmp/dist-test-taskrGEMHD/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskrGEMHD/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskrGEMHD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrGEMHD/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260811 19:02:02.060974 13748 mini_ranger.cc:193] Using host: 127.13.109.20
I20260811 19:02:02.065873 13748 mini_ranger.cc:227] Ranger admin URL: http://127.13.109.20:44991
Aug 11, 2026 7:02:02 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 7:02:02 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 7:02:02 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ranger-admin/ews/webapp
Aug 11, 2026 7:02:02 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 11, 2026 7:02:02 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 11, 2026 7:02:02 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ranger-admin/ews/webapp].
Aug 11, 2026 7:02:02 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260811 19:02:03.258500 13748 mini_ranger.cc:160] Time spent starting Ranger: real 9.177s user 0.006s sys 0.007s
Aug 11, 2026 7:02:23 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 11, 2026 7:02:23 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Aug 11, 2026 7:02:23 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 11, 2026 7:02:23 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 11, 2026 7:02:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 11, 2026 7:02:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 11, 2026 7:02:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 11, 2026 7:02:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 11, 2026 7:02:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 11, 2026 7:02:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 11, 2026 7:02:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 11, 2026 7:02:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 11, 2026 7:02:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 11, 2026 7:02:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 11, 2026 7:02:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 11, 2026 7:02:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 11, 2026 7:02:23 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 11, 2026 7:02:24 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260811 19:02:25.424053 13748 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.13.109.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.13.109.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.13.109.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/krb5kdc/rangerkms_127.13.109.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.13.109.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/krb5kdc/rangerkms_127.13.109.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.13.109.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.13.109.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.13.109.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/krb5kdc/HTTP_127.13.109.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.13.109.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/krb5kdc/HTTP_127.13.109.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 11 19:02:25 dist-test-slave-0kls krb5kdc[26552](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474945, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260811 19:02:25.529865 13748 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260811 19:02:25.541430 13748 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260811 19:02:26.482370 13748 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260811 19:02:26.482488 13748 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ranger-kms
I20260811 19:02:26.482506 13748 mini_ranger_kms.cc:227] Using postgres at 127.13.109.20:35369
2026-08-11 19:02:26,659 [I] DB FLAVOR :POSTGRES
2026-08-11 19:02:26,659 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 19:02:26,659 [I] Checking connection
2026-08-11 19:02:26,660 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrGEMHD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.109.20:35369/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-11 19:02:26,929 [I] connection success
2026-08-11 19:02:26,929 [I] --------- Verifying Ranger DB tables ---------
2026-08-11 19:02:26,929 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-11 19:02:26,929 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrGEMHD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.109.20:35369/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-11 19:02:27,190 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-11 19:02:27,190 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 19:02:27,190 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-11 19:02:27,190 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrGEMHD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.109.20:35369/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-11 19:02:27,461 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260811 19:02:27.964270 13748 mini_ranger_kms.cc:310] Created kms service
I20260811 19:02:28.162149 13748 mini_ranger_kms.cc:326] Created kudu user
I20260811 19:02:28.210696 13748 mini_ranger_kms.cc:343] Created rangerkms user
Aug 11 19:02:28 dist-test-slave-0kls krb5kdc[26552](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474945, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.13.109.20@KRBTEST.COM
I20260811 19:02:28.663699 13748 mini_ranger_kms.cc:384] Added ranger policy
I20260811 19:02:28.663864 13748 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ranger-kms:/tmp/dist-test-taskrGEMHD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrGEMHD/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskrGEMHD/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260811 19:02:28.663885 13748 mini_ranger_kms.cc:240] Using host: 127.13.109.20
I20260811 19:02:28.668542 13748 mini_ranger_kms.cc:276] Ranger KMS PID: 26996
I20260811 19:02:28.668650 13748 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.13.109.20:42179
Aug 11, 2026 7:02:29 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 7:02:29 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 7:02:29 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 11, 2026 7:02:29 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 11, 2026 7:02:30 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ranger-kms/ews/webapp].
Aug 11, 2026 7:02:30 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260811 19:02:31.075142 13748 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.545s user 0.018s sys 0.144s
I20260811 19:02:31.075281 13748 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260811 19:02:31.075378 13748 mini_ranger_kms.cc:401] 127.13.109.20:42179/kms/v1/keys
Aug 11 19:02:31 dist-test-slave-0kls krb5kdc[26552](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786474951, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.13.109.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 11 19:02:31 dist-test-slave-0kls krb5kdc[26552](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474951, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.13.109.20@KRBTEST.COM for HTTP/127.13.109.20@KRBTEST.COM
Aug 11 19:02:32 dist-test-slave-0kls krb5kdc[26552](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474951, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.13.109.20@KRBTEST.COM for HTTP/127.13.109.20@KRBTEST.COM
Aug 11 19:02:34 dist-test-slave-0kls krb5kdc[26552](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474954, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.13.109.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.109.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.109.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab.
Entry for principal kudu/127.13.109.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab.
I20260811 19:02:34.418319 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.62:37499
--webserver_interface=127.13.109.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab
--principal=kudu/127.13.109.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:32997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.109.62:37499
--ranger_config_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.109.20:42179/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:02:34.540205 27054 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:02:34.540496 27054 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:02:34.540536 27054 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:02:34.544735 27054 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:02:34.544806 27054 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:02:34.544826 27054 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:02:34.544883 27054 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:02:34.544912 27054 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:02:34.550727 27054 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:32997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/master-0/wal
--ranger_kms_url=127.13.109.20:42179/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.13.109.62:37499
--ranger_config_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab
--principal=kudu/127.13.109.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.62:37499
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.109.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27054
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:02:34.551904 27054 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:02:34.552992 27054 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:02:34.562556 27060 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:02:34.562644 27061 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:02:34.562678 27063 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:02:34.562752 27054 server_base.cc:1034] running on GCE node
I20260811 19:02:34.563490 27054 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:02:34.564687 27054 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:02:34.565886 27054 hybrid_clock.cc:648] HybridClock initialized: now 1786474954565856 us; error 41 us; skew 500 ppm
Aug 11 19:02:34 dist-test-slave-0kls krb5kdc[26552](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474954, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:02:34.573688 27054 init.cc:377] Logged in from keytab as kudu/127.13.109.62@KRBTEST.COM (short username kudu)
I20260811 19:02:34.574836 27054 webserver.cc:460] Webserver started at http://127.13.109.62:33851/ using document root <none> and password file <none>
I20260811 19:02:34.575397 27054 fs_manager.cc:359] Metadata directory not provided
I20260811 19:02:34.575455 27054 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:02:34.575706 27054 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 19:02:34 dist-test-slave-0kls krb5kdc[26552](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474954, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.62@KRBTEST.COM for HTTP/127.13.109.20@KRBTEST.COM
I20260811 19:02:34.625610 27054 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/master-0/data/instance:
uuid: "368bad7350ee4cf69a7a7ec456f9a088"
format_stamp: "Formatted at 2026-08-11 19:02:34 on dist-test-slave-0kls"
server_key: "5c6d3dbe7a8fdf9d6b90f84b228078e9"
server_key_iv: "cde4b8a855da3e3e6b36cee8d1807553"
server_key_version: "kuduclusterkey@0"
I20260811 19:02:34.626327 27054 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/master-0/wal/instance:
uuid: "368bad7350ee4cf69a7a7ec456f9a088"
format_stamp: "Formatted at 2026-08-11 19:02:34 on dist-test-slave-0kls"
server_key: "5c6d3dbe7a8fdf9d6b90f84b228078e9"
server_key_iv: "cde4b8a855da3e3e6b36cee8d1807553"
server_key_version: "kuduclusterkey@0"
I20260811 19:02:34.630357 27054 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 19:02:34.645103 27071 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:34.646584 27054 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 19:02:34.646730 27054 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/master-0/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/master-0/wal
uuid: "368bad7350ee4cf69a7a7ec456f9a088"
format_stamp: "Formatted at 2026-08-11 19:02:34 on dist-test-slave-0kls"
server_key: "5c6d3dbe7a8fdf9d6b90f84b228078e9"
server_key_iv: "cde4b8a855da3e3e6b36cee8d1807553"
server_key_version: "kuduclusterkey@0"
I20260811 19:02:34.646929 27054 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:02:34.664080 27054 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:02:34.667505 27054 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:02:34.667714 27054 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:02:34.676362 27054 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.62:37499
I20260811 19:02:34.676363 27129 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.62:37499 every 8 connection(s)
I20260811 19:02:34.677564 27054 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
I20260811 19:02:34.681110 27130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:02:34.687420 27130 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 368bad7350ee4cf69a7a7ec456f9a088: Bootstrap starting.
I20260811 19:02:34.687763 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 27054
I20260811 19:02:34.687903 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/master-0/wal/instance
I20260811 19:02:34.688248 13748 external_mini_cluster.cc:1428] Setting key 7647179450a5f5b741bad26108aa52c3
I20260811 19:02:34.690609 27130 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 368bad7350ee4cf69a7a7ec456f9a088: Neither blocks nor log segments found. Creating new log.
I20260811 19:02:34.691470 27130 log.cc:824] T 00000000000000000000000000000000 P 368bad7350ee4cf69a7a7ec456f9a088: Log is configured to *not* fsync() on all Append() calls
I20260811 19:02:34.693996 27130 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 368bad7350ee4cf69a7a7ec456f9a088: No bootstrap required, opened a new log
I20260811 19:02:34.698797 27130 raft_consensus.cc:348] T 00000000000000000000000000000000 P 368bad7350ee4cf69a7a7ec456f9a088 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "368bad7350ee4cf69a7a7ec456f9a088" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 37499 } }
I20260811 19:02:34.699174 27130 raft_consensus.cc:374] T 00000000000000000000000000000000 P 368bad7350ee4cf69a7a7ec456f9a088 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:02:34.699260 27130 raft_consensus.cc:729] T 00000000000000000000000000000000 P 368bad7350ee4cf69a7a7ec456f9a088 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 368bad7350ee4cf69a7a7ec456f9a088, State: Initialized, Role: FOLLOWER
Aug 11 19:02:34 dist-test-slave-0kls krb5kdc[26552](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474954, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
I20260811 19:02:34.699867 27130 consensus_queue.cc:260] T 00000000000000000000000000000000 P 368bad7350ee4cf69a7a7ec456f9a088 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "368bad7350ee4cf69a7a7ec456f9a088" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 37499 } }
I20260811 19:02:34.700037 27130 raft_consensus.cc:388] T 00000000000000000000000000000000 P 368bad7350ee4cf69a7a7ec456f9a088 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:02:34.700148 27130 raft_consensus.cc:482] T 00000000000000000000000000000000 P 368bad7350ee4cf69a7a7ec456f9a088 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:02:34.700294 27130 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 368bad7350ee4cf69a7a7ec456f9a088 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:02:34.701747 27130 raft_consensus.cc:504] T 00000000000000000000000000000000 P 368bad7350ee4cf69a7a7ec456f9a088 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "368bad7350ee4cf69a7a7ec456f9a088" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 37499 } }
I20260811 19:02:34.702224 27130 leader_election.cc:302] T 00000000000000000000000000000000 P 368bad7350ee4cf69a7a7ec456f9a088 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 368bad7350ee4cf69a7a7ec456f9a088; no voters:
I20260811 19:02:34.702672 27130 leader_election.cc:288] T 00000000000000000000000000000000 P 368bad7350ee4cf69a7a7ec456f9a088 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:02:34.702787 27135 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 368bad7350ee4cf69a7a7ec456f9a088 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:02:34.703821 27135 raft_consensus.cc:686] T 00000000000000000000000000000000 P 368bad7350ee4cf69a7a7ec456f9a088 [term 1 LEADER]: Becoming Leader. State: Replica: 368bad7350ee4cf69a7a7ec456f9a088, State: Running, Role: LEADER
I20260811 19:02:34.704321 27130 sys_catalog.cc:564] T 00000000000000000000000000000000 P 368bad7350ee4cf69a7a7ec456f9a088 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:02:34.704288 27135 consensus_queue.cc:237] T 00000000000000000000000000000000 P 368bad7350ee4cf69a7a7ec456f9a088 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "368bad7350ee4cf69a7a7ec456f9a088" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 37499 } }
I20260811 19:02:34.706300 27133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:34.689816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47016 (local address 127.13.109.62:37499)
0811 19:02:34.690384 (+ 568us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:34.690404 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:34.690454 (+ 50us) server_negotiation.cc:407] Connection header received
0811 19:02:34.691392 (+ 938us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:34.691420 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:34.691892 (+ 472us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:34.692309 (+ 417us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:02:34.693262 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:34.694829 (+ 1567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:34.696209 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:34.696948 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:34.697131 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:34.700408 (+ 3277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:02:34.700415 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:02:34.700437 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:02:34.700480 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:02:34.702758 (+ 2278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:34.703297 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:34.703304 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:34.703311 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:34.703453 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:34.703752 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:34.703756 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:34.703757 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:34.704305 (+ 548us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:02:34.704508 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:34.704810 (+ 302us) server_negotiation.cc:299] Negotiation successful
0811 19:02:34.705118 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":383,"thread_start_us":167,"threads_started":1}
I20260811 19:02:34.706614 27135 sys_catalog.cc:455] T 00000000000000000000000000000000 P 368bad7350ee4cf69a7a7ec456f9a088 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 368bad7350ee4cf69a7a7ec456f9a088. Latest consensus state: current_term: 1 leader_uuid: "368bad7350ee4cf69a7a7ec456f9a088" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "368bad7350ee4cf69a7a7ec456f9a088" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 37499 } } }
I20260811 19:02:34.706736 27135 sys_catalog.cc:458] T 00000000000000000000000000000000 P 368bad7350ee4cf69a7a7ec456f9a088 [sys.catalog]: This master's current role is: LEADER
I20260811 19:02:34.707173 27152 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:02:34.707531 27130 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 19:02:34.709031 27136 sys_catalog.cc:455] T 00000000000000000000000000000000 P 368bad7350ee4cf69a7a7ec456f9a088 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "368bad7350ee4cf69a7a7ec456f9a088" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "368bad7350ee4cf69a7a7ec456f9a088" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 37499 } } }
I20260811 19:02:34.709203 27136 sys_catalog.cc:458] T 00000000000000000000000000000000 P 368bad7350ee4cf69a7a7ec456f9a088 [sys.catalog]: This master's current role is: LEADER
I20260811 19:02:34.715417 27152 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:02:34.725070 27152 catalog_manager.cc:1347] Generated new cluster ID: ff1958b2d1ae4826a2dc900d870db28b
I20260811 19:02:34.725246 27152 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:02:34.739997 27152 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:02:34.741187 27152 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:02:34.755555 27152 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 368bad7350ee4cf69a7a7ec456f9a088: Generated new TSK 0
I20260811 19:02:34.756608 27152 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.109.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.109.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.109.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.1.keytab.
Entry for principal kudu/127.13.109.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.1.keytab.
I20260811 19:02:35.490362 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.109.20:42179/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.1:0
--local_ip_for_outbound_sockets=127.13.109.1
--webserver_interface=127.13.109.1
--webserver_port=0
--tserver_master_addrs=127.13.109.62:37499
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.1.keytab
--principal=kudu/127.13.109.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:32997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:02:35.656992 27182 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:02:35.657335 27182 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:02:35.657413 27182 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:02:35.662660 27182 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:02:35.662752 27182 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:02:35.662849 27182 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.1
I20260811 19:02:35.671082 27182 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:32997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.13.109.20:42179/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.1.keytab
--principal=kudu/127.13.109.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.109.1
--webserver_port=0
--tserver_master_addrs=127.13.109.62:37499
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27182
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.1
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:02:35.672722 27182 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:02:35.674135 27182 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:02:35.687086 27190 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:02:35.687441 27192 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:02:35.687673 27194 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:02:35.687953 27182 server_base.cc:1034] running on GCE node
I20260811 19:02:35.688395 27182 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:02:35.689105 27182 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:02:35.690310 27182 hybrid_clock.cc:648] HybridClock initialized: now 1786474955690272 us; error 45 us; skew 500 ppm
Aug 11 19:02:35 dist-test-slave-0kls krb5kdc[26552](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474955, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:02:35.693292 27182 init.cc:377] Logged in from keytab as kudu/127.13.109.1@KRBTEST.COM (short username kudu)
I20260811 19:02:35.694445 27182 webserver.cc:460] Webserver started at http://127.13.109.1:40087/ using document root <none> and password file <none>
I20260811 19:02:35.695057 27182 fs_manager.cc:359] Metadata directory not provided
I20260811 19:02:35.695156 27182 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:02:35.695427 27182 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 19:02:35 dist-test-slave-0kls krb5kdc[26552](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474955, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.1@KRBTEST.COM for HTTP/127.13.109.20@KRBTEST.COM
I20260811 19:02:35.712642 27182 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ts-0/data/instance:
uuid: "a76ab6148c384490b58109484e6fac3f"
format_stamp: "Formatted at 2026-08-11 19:02:35 on dist-test-slave-0kls"
server_key: "a0df5ea4071e2c4a1c673d16b92a2464"
server_key_iv: "9215e0374e35ec8fdd96107e77a2617e"
server_key_version: "kuduclusterkey@0"
I20260811 19:02:35.714712 27182 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ts-0/wal/instance:
uuid: "a76ab6148c384490b58109484e6fac3f"
format_stamp: "Formatted at 2026-08-11 19:02:35 on dist-test-slave-0kls"
server_key: "a0df5ea4071e2c4a1c673d16b92a2464"
server_key_iv: "9215e0374e35ec8fdd96107e77a2617e"
server_key_version: "kuduclusterkey@0"
I20260811 19:02:35.719229 27182 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 19:02:35.742249 27202 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:35.748382 27182 fs_manager.cc:725] Time spent opening block manager: real 0.014s user 0.001s sys 0.000s
I20260811 19:02:35.748559 27182 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ts-0/wal
uuid: "a76ab6148c384490b58109484e6fac3f"
format_stamp: "Formatted at 2026-08-11 19:02:35 on dist-test-slave-0kls"
server_key: "a0df5ea4071e2c4a1c673d16b92a2464"
server_key_iv: "9215e0374e35ec8fdd96107e77a2617e"
server_key_version: "kuduclusterkey@0"
I20260811 19:02:35.748716 27182 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:02:35.780089 27182 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:02:35.783999 27182 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:02:35.784281 27182 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:02:35.785064 27182 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:02:35.786235 27182 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:02:35.786352 27182 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:35.786480 27182 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:02:35.786778 27182 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:35.798568 27182 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.1:43647
I20260811 19:02:35.800189 27332 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.1:43647 every 8 connection(s)
I20260811 19:02:35.800583 27182 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
I20260811 19:02:35.807581 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 27182
I20260811 19:02:35.807722 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ts-0/wal/instance
I20260811 19:02:35.808053 13748 external_mini_cluster.cc:1428] Setting key 8af5748e2d340660364d173c93000e4e
Aug 11 19:02:35 dist-test-slave-0kls krb5kdc[26552](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474955, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.1@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.109.2@KRBTEST.COM; defaulting to no policy
I20260811 19:02:35.828060 27337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:35.804924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.1:51715 (local address 127.13.109.62:37499)
0811 19:02:35.814878 (+ 9954us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:35.814884 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:35.814901 (+ 17us) server_negotiation.cc:407] Connection header received
0811 19:02:35.814948 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:35.814951 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:35.815004 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:35.815108 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:02:35.816749 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:35.817601 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:35.819471 (+ 1870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:35.819871 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:35.819955 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:35.823291 (+ 3336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:02:35.823295 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:02:35.823298 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:02:35.823330 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:02:35.825286 (+ 1956us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:35.826011 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:35.826015 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:35.826017 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:35.826115 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:35.826664 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:35.826668 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:35.826670 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:35.826905 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:02:35.826986 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:35.827723 (+ 737us) server_negotiation.cc:299] Negotiation successful
0811 19:02:35.827868 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9859,"thread_start_us":95,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.109.2@KRBTEST.COM" created.
I20260811 19:02:35.829833 27336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:35.803747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:37499 (local address 127.13.109.1:51715)
0811 19:02:35.812986 (+ 9239us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:35.813086 (+ 100us) client_negotiation.cc:174] Beginning negotiation
0811 19:02:35.814335 (+ 1249us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:02:35.815390 (+ 1055us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:02:35.815402 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:02:35.815822 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:02:35.816562 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:35.816584 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:35.817808 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:35.817812 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:35.819303 (+ 1491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:35.819317 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:35.820198 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:35.820202 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:35.820432 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:35.821160 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:02:35.821183 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:02:35.823059 (+ 1876us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:02:35.825441 (+ 2382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:02:35.825447 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:02:35.825468 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:02:35.825881 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:02:35.826236 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:02:35.826239 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:02:35.826242 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:02:35.826532 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:02:35.827055 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:02:35.827062 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:02:35.827410 (+ 348us) client_negotiation.cc:769] Sending connection context
0811 19:02:35.828322 (+ 912us) client_negotiation.cc:240] Negotiation successful
0811 19:02:35.828713 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8870,"thread_start_us":150,"threads_started":1}
I20260811 19:02:35.831568 27334 heartbeater.cc:347] Connected to a master server at 127.13.109.62:37499
I20260811 19:02:35.831854 27334 heartbeater.cc:464] Registering TS with master...
I20260811 19:02:35.832477 27334 heartbeater.cc:511] Master 127.13.109.62:37499 requested a full tablet report, sending...
I20260811 19:02:35.834427 27090 ts_manager.cc:194] Registered new tserver with Master: a76ab6148c384490b58109484e6fac3f (127.13.109.1:43647)
I20260811 19:02:35.836071 27090 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.109.1@KRBTEST.COM'} at 127.13.109.1:51715
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.109.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.2.keytab.
Entry for principal kudu/127.13.109.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.2.keytab.
I20260811 19:02:35.854357 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.109.20:42179/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.2:0
--local_ip_for_outbound_sockets=127.13.109.2
--webserver_interface=127.13.109.2
--webserver_port=0
--tserver_master_addrs=127.13.109.62:37499
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.2.keytab
--principal=kudu/127.13.109.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:32997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 19:02:35 dist-test-slave-0kls krb5kdc[26552](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786474955, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260811 19:02:36.002192 27341 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:02:36.002681 27341 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:02:36.002772 27341 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:02:36.009850 27341 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:02:36.009997 27341 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:02:36.010147 27341 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.2
I20260811 19:02:36.016582 27341 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:32997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.13.109.20:42179/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.2.keytab
--principal=kudu/127.13.109.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.109.2
--webserver_port=0
--tserver_master_addrs=127.13.109.62:37499
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27341
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.2
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:02:36.017773 27341 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:02:36.018889 27341 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:02:36.030326 27348 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:02:36.030619 27351 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:02:36.030292 27349 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:02:36.031318 27341 server_base.cc:1034] running on GCE node
I20260811 19:02:36.031804 27341 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:02:36.032691 27341 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:02:36.034040 27341 hybrid_clock.cc:648] HybridClock initialized: now 1786474956033968 us; error 97 us; skew 500 ppm
Aug 11 19:02:36 dist-test-slave-0kls krb5kdc[26552](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474956, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:02:36.037691 27341 init.cc:377] Logged in from keytab as kudu/127.13.109.2@KRBTEST.COM (short username kudu)
I20260811 19:02:36.038882 27341 webserver.cc:460] Webserver started at http://127.13.109.2:38199/ using document root <none> and password file <none>
I20260811 19:02:36.039541 27341 fs_manager.cc:359] Metadata directory not provided
I20260811 19:02:36.039642 27341 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:02:36.039909 27341 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 19:02:36 dist-test-slave-0kls krb5kdc[26552](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474956, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.2@KRBTEST.COM for HTTP/127.13.109.20@KRBTEST.COM
I20260811 19:02:36.054567 27341 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ts-1/data/instance:
uuid: "3eb0da02d24c4b6cab01b7a221c5566c"
format_stamp: "Formatted at 2026-08-11 19:02:36 on dist-test-slave-0kls"
server_key: "b8da9910129cf168a0d20d5ec3e37180"
server_key_iv: "3170d394ef4b1d5052b9ef855b705a93"
server_key_version: "kuduclusterkey@0"
I20260811 19:02:36.055495 27341 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ts-1/wal/instance:
uuid: "3eb0da02d24c4b6cab01b7a221c5566c"
format_stamp: "Formatted at 2026-08-11 19:02:36 on dist-test-slave-0kls"
server_key: "b8da9910129cf168a0d20d5ec3e37180"
server_key_iv: "3170d394ef4b1d5052b9ef855b705a93"
server_key_version: "kuduclusterkey@0"
I20260811 19:02:36.063374 27341 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.004s sys 0.000s
I20260811 19:02:36.081583 27359 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:36.083498 27341 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260811 19:02:36.083670 27341 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ts-1/wal
uuid: "3eb0da02d24c4b6cab01b7a221c5566c"
format_stamp: "Formatted at 2026-08-11 19:02:36 on dist-test-slave-0kls"
server_key: "b8da9910129cf168a0d20d5ec3e37180"
server_key_iv: "3170d394ef4b1d5052b9ef855b705a93"
server_key_version: "kuduclusterkey@0"
I20260811 19:02:36.083845 27341 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:02:36.111425 27341 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:02:36.115451 27341 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:02:36.115748 27341 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:02:36.116515 27341 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:02:36.117717 27341 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:02:36.117878 27341 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:36.117992 27341 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:02:36.118059 27341 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:36.132447 27341 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.2:39067
I20260811 19:02:36.133873 27341 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
I20260811 19:02:36.135694 27489 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.2:39067 every 8 connection(s)
I20260811 19:02:36.143373 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 27341
I20260811 19:02:36.143520 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ts-1/wal/instance
I20260811 19:02:36.143837 13748 external_mini_cluster.cc:1428] Setting key 92f0b33a38b6db428af82774e9c95baa
Aug 11 19:02:36 dist-test-slave-0kls krb5kdc[26552](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474956, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.2@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.109.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.109.3@KRBTEST.COM" created.
I20260811 19:02:36.161557 27337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:36.139756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.2:53887 (local address 127.13.109.62:37499)
0811 19:02:36.139902 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:36.139906 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:36.146274 (+ 6368us) server_negotiation.cc:407] Connection header received
0811 19:02:36.147842 (+ 1568us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:36.147848 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:36.147919 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:36.147981 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:02:36.149934 (+ 1953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:36.150808 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:36.152766 (+ 1958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:36.153230 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:36.153328 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:36.156967 (+ 3639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:02:36.156972 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:02:36.156975 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:02:36.157013 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:02:36.158755 (+ 1742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:36.159429 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:36.159432 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:36.159434 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:36.159529 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:36.160001 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:36.160005 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:36.160006 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:36.160264 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:02:36.160358 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:36.161131 (+ 773us) server_negotiation.cc:299] Negotiation successful
0811 19:02:36.161302 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 19:02:36.163733 27493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:36.140221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:37499 (local address 127.13.109.2:53887)
0811 19:02:36.146085 (+ 5864us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:36.146163 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0811 19:02:36.147541 (+ 1378us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:02:36.148480 (+ 939us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:02:36.148491 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:02:36.148917 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:02:36.149714 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:36.149741 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:36.150993 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:36.150999 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:36.152591 (+ 1592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:36.152609 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:36.153587 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:36.153594 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:36.153892 (+ 298us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:36.154657 (+ 765us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:02:36.154685 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:02:36.156709 (+ 2024us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:02:36.158889 (+ 2180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:02:36.158931 (+ 42us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:02:36.158955 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:02:36.159273 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:02:36.159650 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:02:36.159653 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:02:36.159654 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:02:36.159878 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:02:36.160432 (+ 554us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:02:36.160440 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:02:36.160834 (+ 394us) client_negotiation.cc:769] Sending connection context
0811 19:02:36.162087 (+ 1253us) client_negotiation.cc:240] Negotiation successful
0811 19:02:36.162498 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5588,"thread_start_us":168,"threads_started":1}
I20260811 19:02:36.165572 27490 heartbeater.cc:347] Connected to a master server at 127.13.109.62:37499
I20260811 19:02:36.165911 27490 heartbeater.cc:464] Registering TS with master...
I20260811 19:02:36.166467 27490 heartbeater.cc:511] Master 127.13.109.62:37499 requested a full tablet report, sending...
I20260811 19:02:36.167980 27090 ts_manager.cc:194] Registered new tserver with Master: 3eb0da02d24c4b6cab01b7a221c5566c (127.13.109.2:39067)
I20260811 19:02:36.168788 27090 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.109.2@KRBTEST.COM'} at 127.13.109.2:53887
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.109.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.3.keytab.
Entry for principal kudu/127.13.109.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.3.keytab.
I20260811 19:02:36.182310 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.109.20:42179/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.3:0
--local_ip_for_outbound_sockets=127.13.109.3
--webserver_interface=127.13.109.3
--webserver_port=0
--tserver_master_addrs=127.13.109.62:37499
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.3.keytab
--principal=kudu/127.13.109.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:32997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:02:36.352449 27496 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:02:36.352981 27496 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:02:36.353120 27496 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:02:36.359575 27496 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:02:36.359773 27496 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:02:36.359923 27496 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.3
I20260811 19:02:36.368431 27496 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:32997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.13.109.20:42179/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.3.keytab
--principal=kudu/127.13.109.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.109.3
--webserver_port=0
--tserver_master_addrs=127.13.109.62:37499
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27496
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.3
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:02:36.370361 27496 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:02:36.371960 27496 file_cache.cc:493] Constructed file cache file cache with capacity 419430
Aug 11 19:02:36 dist-test-slave-0kls krb5kdc[26552](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474955, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.62@KRBTEST.COM for HTTP/127.13.109.20@KRBTEST.COM
W20260811 19:02:36.385473 27504 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:02:36.385648 27502 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:02:36.386093 27501 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:02:36.386137 27496 server_base.cc:1034] running on GCE node
I20260811 19:02:36.386732 27496 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:02:36.387549 27496 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:02:36.388778 27496 hybrid_clock.cc:648] HybridClock initialized: now 1786474956388713 us; error 85 us; skew 500 ppm
Aug 11 19:02:36 dist-test-slave-0kls krb5kdc[26552](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474956, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:02:36.392622 27496 init.cc:377] Logged in from keytab as kudu/127.13.109.3@KRBTEST.COM (short username kudu)
I20260811 19:02:36.393740 27496 webserver.cc:460] Webserver started at http://127.13.109.3:42993/ using document root <none> and password file <none>
I20260811 19:02:36.394568 27496 fs_manager.cc:359] Metadata directory not provided
I20260811 19:02:36.394724 27496 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:02:36.395044 27496 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 19:02:36 dist-test-slave-0kls krb5kdc[26552](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474956, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.3@KRBTEST.COM for HTTP/127.13.109.20@KRBTEST.COM
I20260811 19:02:36.423259 27496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ts-2/data/instance:
uuid: "df2d01a188784505b8dd3d9e81b6a07d"
format_stamp: "Formatted at 2026-08-11 19:02:36 on dist-test-slave-0kls"
server_key: "921be8086db079a6e791f97db77638ad"
server_key_iv: "049b2f99a3765f1e2ea8fd6365f81646"
server_key_version: "kuduclusterkey@0"
I20260811 19:02:36.424255 27496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ts-2/wal/instance:
uuid: "df2d01a188784505b8dd3d9e81b6a07d"
format_stamp: "Formatted at 2026-08-11 19:02:36 on dist-test-slave-0kls"
server_key: "921be8086db079a6e791f97db77638ad"
server_key_iv: "049b2f99a3765f1e2ea8fd6365f81646"
server_key_version: "kuduclusterkey@0"
I20260811 19:02:36.439026 27496 fs_manager.cc:691] Time spent creating directory manager: real 0.014s user 0.001s sys 0.004s
I20260811 19:02:36.482303 27511 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:36.484462 27496 fs_manager.cc:725] Time spent opening block manager: real 0.019s user 0.004s sys 0.000s
I20260811 19:02:36.484649 27496 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ts-2/wal
uuid: "df2d01a188784505b8dd3d9e81b6a07d"
format_stamp: "Formatted at 2026-08-11 19:02:36 on dist-test-slave-0kls"
server_key: "921be8086db079a6e791f97db77638ad"
server_key_iv: "049b2f99a3765f1e2ea8fd6365f81646"
server_key_version: "kuduclusterkey@0"
I20260811 19:02:36.484813 27496 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:02:36.520682 27496 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:02:36.524917 27496 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:02:36.525170 27496 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:02:36.526080 27496 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:02:36.527362 27496 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:02:36.527436 27496 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:36.527501 27496 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:02:36.527601 27496 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260811 19:02:36.542629 27496 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.3:34487
I20260811 19:02:36.543982 27496 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
I20260811 19:02:36.547398 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 27496
I20260811 19:02:36.547555 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ts-2/wal/instance
I20260811 19:02:36.547883 13748 external_mini_cluster.cc:1428] Setting key b831c222479a538ccdbbd3579d5c1287
I20260811 19:02:36.551453 27638 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.3:34487 every 8 connection(s)
Aug 11 19:02:36 dist-test-slave-0kls krb5kdc[26552](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474956, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.3@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
I20260811 19:02:36.575299 27337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:36.553986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.3:45637 (local address 127.13.109.62:37499)
0811 19:02:36.554145 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:36.554150 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:36.556437 (+ 2287us) server_negotiation.cc:407] Connection header received
0811 19:02:36.557816 (+ 1379us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:36.557820 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:36.557888 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:36.557949 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:02:36.559825 (+ 1876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:36.560684 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:36.564546 (+ 3862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:36.564926 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:36.565012 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:36.568559 (+ 3547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:02:36.568564 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:02:36.568566 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:02:36.568602 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:02:36.570486 (+ 1884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:36.571265 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:36.571269 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:36.571271 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:36.571369 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:36.571857 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:36.571861 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:36.571863 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:36.572111 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:02:36.572188 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:36.574969 (+ 2781us) server_negotiation.cc:299] Negotiation successful
0811 19:02:36.575118 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 19:02:36.577474 27641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:36.554438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:37499 (local address 127.13.109.3:45637)
0811 19:02:36.556251 (+ 1813us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:36.556317 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0811 19:02:36.557489 (+ 1172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:02:36.558420 (+ 931us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:02:36.558430 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:02:36.558850 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:02:36.559587 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:36.559610 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:36.562766 (+ 3156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:36.562770 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:36.564368 (+ 1598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:36.564386 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:36.565272 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:36.565276 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:36.565504 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:36.566332 (+ 828us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:02:36.566357 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:02:36.568299 (+ 1942us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:02:36.570676 (+ 2377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:02:36.570687 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:02:36.570716 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:02:36.571126 (+ 410us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:02:36.571485 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:02:36.571488 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:02:36.571491 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:02:36.571718 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:02:36.572276 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:02:36.572284 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:02:36.574619 (+ 2335us) client_negotiation.cc:769] Sending connection context
0811 19:02:36.575804 (+ 1185us) client_negotiation.cc:240] Negotiation successful
0811 19:02:36.576259 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1554,"thread_start_us":196,"threads_started":1}
I20260811 19:02:36.579226 27639 heartbeater.cc:347] Connected to a master server at 127.13.109.62:37499
I20260811 19:02:36.579502 27639 heartbeater.cc:464] Registering TS with master...
I20260811 19:02:36.580199 27639 heartbeater.cc:511] Master 127.13.109.62:37499 requested a full tablet report, sending...
I20260811 19:02:36.581542 27090 ts_manager.cc:194] Registered new tserver with Master: df2d01a188784505b8dd3d9e81b6a07d (127.13.109.3:34487)
I20260811 19:02:36.582391 27090 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.109.3@KRBTEST.COM'} at 127.13.109.3:45637
I20260811 19:02:36.586309 13748 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 19:02:36.587144 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 27182
I20260811 19:02:36.609273 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 27341
I20260811 19:02:36.619875 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 27496
I20260811 19:02:36.630148 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 27054
I20260811 19:02:36.640211 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.62:37499
--webserver_interface=127.13.109.62
--webserver_port=33851
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab
--principal=kudu/127.13.109.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:32997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.109.62:37499
--ranger_config_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.109.20:42179/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.13.109.20:42179/kms with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:02:36.764096 27642 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:02:36.764487 27642 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:02:36.764577 27642 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:02:36.769084 27642 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:02:36.769198 27642 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:02:36.769228 27642 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:02:36.769258 27642 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:02:36.769294 27642 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:02:36.775563 27642 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:32997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.13.109.20:42179/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.13.109.62:37499
--ranger_config_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab
--principal=kudu/127.13.109.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.62:37499
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.109.62
--webserver_port=33851
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27642
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:02:36.776870 27642 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:02:36.778113 27642 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:02:36.787997 27647 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:02:36.788018 27650 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:02:36.788007 27648 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:02:36.788503 27642 server_base.cc:1034] running on GCE node
I20260811 19:02:36.789215 27642 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:02:36.790588 27642 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:02:36.791802 27642 hybrid_clock.cc:648] HybridClock initialized: now 1786474956791772 us; error 59 us; skew 500 ppm
Aug 11 19:02:36 dist-test-slave-0kls krb5kdc[26552](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474956, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:02:36.795250 27642 init.cc:377] Logged in from keytab as kudu/127.13.109.62@KRBTEST.COM (short username kudu)
I20260811 19:02:36.796454 27642 webserver.cc:460] Webserver started at http://127.13.109.62:33851/ using document root <none> and password file <none>
I20260811 19:02:36.797120 27642 fs_manager.cc:359] Metadata directory not provided
I20260811 19:02:36.797206 27642 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 19:02:36 dist-test-slave-0kls krb5kdc[26552](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474956, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.62@KRBTEST.COM for HTTP/127.13.109.20@KRBTEST.COM
I20260811 19:02:36.886081 27642 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 19:02:36.888590 27659 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:36.890100 27642 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 19:02:36.890254 27642 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/master-0/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/master-0/wal
uuid: "368bad7350ee4cf69a7a7ec456f9a088"
format_stamp: "Formatted at 2026-08-11 19:02:34 on dist-test-slave-0kls"
server_key: "5c6d3dbe7a8fdf9d6b90f84b228078e9"
server_key_iv: "cde4b8a855da3e3e6b36cee8d1807553"
server_key_version: "kuduclusterkey@0"
I20260811 19:02:36.890843 27642 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:02:36.903757 27642 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:02:36.907058 27642 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:02:36.907294 27642 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:02:36.916174 27642 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.62:37499
I20260811 19:02:36.916184 27717 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.62:37499 every 8 connection(s)
I20260811 19:02:36.917393 27642 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
I20260811 19:02:36.919931 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 27642
I20260811 19:02:36.920461 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.109.20:42179/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.1:43647
--local_ip_for_outbound_sockets=127.13.109.1
--tserver_master_addrs=127.13.109.62:37499
--webserver_port=40087
--webserver_interface=127.13.109.1
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.1.keytab
--principal=kudu/127.13.109.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:32997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.13.109.20:42179/kms with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 19:02:36.922485 27718 sys_catalog.cc:263] Verifying existing consensus state
I20260811 19:02:36.925056 27718 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 368bad7350ee4cf69a7a7ec456f9a088: Bootstrap starting.
I20260811 19:02:36.949877 27718 log.cc:824] T 00000000000000000000000000000000 P 368bad7350ee4cf69a7a7ec456f9a088: Log is configured to *not* fsync() on all Append() calls
I20260811 19:02:36.958772 27718 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 368bad7350ee4cf69a7a7ec456f9a088: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 19:02:36.959297 27718 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 368bad7350ee4cf69a7a7ec456f9a088: Bootstrap complete.
I20260811 19:02:36.965058 27718 raft_consensus.cc:348] T 00000000000000000000000000000000 P 368bad7350ee4cf69a7a7ec456f9a088 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "368bad7350ee4cf69a7a7ec456f9a088" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 37499 } }
I20260811 19:02:36.965628 27718 raft_consensus.cc:729] T 00000000000000000000000000000000 P 368bad7350ee4cf69a7a7ec456f9a088 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 368bad7350ee4cf69a7a7ec456f9a088, State: Initialized, Role: FOLLOWER
I20260811 19:02:36.966254 27718 consensus_queue.cc:260] T 00000000000000000000000000000000 P 368bad7350ee4cf69a7a7ec456f9a088 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "368bad7350ee4cf69a7a7ec456f9a088" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 37499 } }
I20260811 19:02:36.966413 27718 raft_consensus.cc:388] T 00000000000000000000000000000000 P 368bad7350ee4cf69a7a7ec456f9a088 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:02:36.966485 27718 raft_consensus.cc:482] T 00000000000000000000000000000000 P 368bad7350ee4cf69a7a7ec456f9a088 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:02:36.966583 27718 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 368bad7350ee4cf69a7a7ec456f9a088 [term 1 FOLLOWER]: Advancing to term 2
I20260811 19:02:36.969434 27718 raft_consensus.cc:504] T 00000000000000000000000000000000 P 368bad7350ee4cf69a7a7ec456f9a088 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "368bad7350ee4cf69a7a7ec456f9a088" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 37499 } }
I20260811 19:02:36.969862 27718 leader_election.cc:302] T 00000000000000000000000000000000 P 368bad7350ee4cf69a7a7ec456f9a088 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 368bad7350ee4cf69a7a7ec456f9a088; no voters:
I20260811 19:02:36.970270 27718 leader_election.cc:288] T 00000000000000000000000000000000 P 368bad7350ee4cf69a7a7ec456f9a088 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 19:02:36.970438 27724 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 368bad7350ee4cf69a7a7ec456f9a088 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 19:02:36.970745 27724 raft_consensus.cc:686] T 00000000000000000000000000000000 P 368bad7350ee4cf69a7a7ec456f9a088 [term 2 LEADER]: Becoming Leader. State: Replica: 368bad7350ee4cf69a7a7ec456f9a088, State: Running, Role: LEADER
I20260811 19:02:36.971144 27724 consensus_queue.cc:237] T 00000000000000000000000000000000 P 368bad7350ee4cf69a7a7ec456f9a088 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "368bad7350ee4cf69a7a7ec456f9a088" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 37499 } }
I20260811 19:02:36.971671 27718 sys_catalog.cc:564] T 00000000000000000000000000000000 P 368bad7350ee4cf69a7a7ec456f9a088 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:02:36.972771 27726 sys_catalog.cc:455] T 00000000000000000000000000000000 P 368bad7350ee4cf69a7a7ec456f9a088 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 368bad7350ee4cf69a7a7ec456f9a088. Latest consensus state: current_term: 2 leader_uuid: "368bad7350ee4cf69a7a7ec456f9a088" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "368bad7350ee4cf69a7a7ec456f9a088" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 37499 } } }
I20260811 19:02:36.972924 27726 sys_catalog.cc:458] T 00000000000000000000000000000000 P 368bad7350ee4cf69a7a7ec456f9a088 [sys.catalog]: This master's current role is: LEADER
I20260811 19:02:36.973222 27725 sys_catalog.cc:455] T 00000000000000000000000000000000 P 368bad7350ee4cf69a7a7ec456f9a088 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "368bad7350ee4cf69a7a7ec456f9a088" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "368bad7350ee4cf69a7a7ec456f9a088" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 37499 } } }
I20260811 19:02:36.973302 27725 sys_catalog.cc:458] T 00000000000000000000000000000000 P 368bad7350ee4cf69a7a7ec456f9a088 [sys.catalog]: This master's current role is: LEADER
I20260811 19:02:36.973328 27734 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:02:36.976732 27734 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:02:36.977299 27718 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 19:02:36.977471 27734 catalog_manager.cc:1259] Loaded cluster ID: ff1958b2d1ae4826a2dc900d870db28b
I20260811 19:02:36.977521 27734 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:02:36.979977 27734 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:02:36.980672 27734 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 368bad7350ee4cf69a7a7ec456f9a088: Loaded TSK: 0
I20260811 19:02:36.981977 27734 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 19:02:37.086404 27720 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:02:37.086812 27720 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:02:37.086885 27720 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:02:37.092986 27720 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:02:37.093086 27720 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:02:37.093187 27720 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.1
I20260811 19:02:37.101385 27720 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:32997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.13.109.20:42179/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.1.keytab
--principal=kudu/127.13.109.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.1:43647
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.109.1
--webserver_port=40087
--tserver_master_addrs=127.13.109.62:37499
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27720
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.1
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:02:37.102936 27720 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:02:37.104293 27720 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:02:37.117314 27764 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:02:37.117493 27761 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:02:37.117827 27760 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:02:37.118088 27720 server_base.cc:1034] running on GCE node
I20260811 19:02:37.118588 27720 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:02:37.119495 27720 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:02:37.120661 27720 hybrid_clock.cc:648] HybridClock initialized: now 1786474957120633 us; error 52 us; skew 500 ppm
Aug 11 19:02:37 dist-test-slave-0kls krb5kdc[26552](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474957, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:02:37.124064 27720 init.cc:377] Logged in from keytab as kudu/127.13.109.1@KRBTEST.COM (short username kudu)
I20260811 19:02:37.125150 27720 webserver.cc:460] Webserver started at http://127.13.109.1:40087/ using document root <none> and password file <none>
I20260811 19:02:37.125931 27720 fs_manager.cc:359] Metadata directory not provided
I20260811 19:02:37.126039 27720 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 19:02:37 dist-test-slave-0kls krb5kdc[26552](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474957, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.1@KRBTEST.COM for HTTP/127.13.109.20@KRBTEST.COM
I20260811 19:02:37.164319 27720 fs_manager.cc:709] Time spent opening directory manager: real 0.010s user 0.003s sys 0.000s
I20260811 19:02:37.168030 27772 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:37.169775 27720 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260811 19:02:37.169927 27720 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ts-0/wal
uuid: "a76ab6148c384490b58109484e6fac3f"
format_stamp: "Formatted at 2026-08-11 19:02:35 on dist-test-slave-0kls"
server_key: "a0df5ea4071e2c4a1c673d16b92a2464"
server_key_iv: "9215e0374e35ec8fdd96107e77a2617e"
server_key_version: "kuduclusterkey@0"
I20260811 19:02:37.170679 27720 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:02:37.185190 27720 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:02:37.192595 27720 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:02:37.192902 27720 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:02:37.193749 27720 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:02:37.195115 27720 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:02:37.195254 27720 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:37.195446 27720 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:02:37.195520 27720 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:37.208767 27720 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.1:43647
I20260811 19:02:37.209059 27900 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.1:43647 every 8 connection(s)
I20260811 19:02:37.210286 27720 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
I20260811 19:02:37.212852 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 27720
I20260811 19:02:37.213403 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.109.20:42179/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.2:39067
--local_ip_for_outbound_sockets=127.13.109.2
--tserver_master_addrs=127.13.109.62:37499
--webserver_port=38199
--webserver_interface=127.13.109.2
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.2.keytab
--principal=kudu/127.13.109.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:32997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.13.109.20:42179/kms with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 19:02:37 dist-test-slave-0kls krb5kdc[26552](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474957, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.1@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
I20260811 19:02:37.246840 27904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:37.213071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:37499 (local address 127.13.109.1:41253)
0811 19:02:37.213701 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:37.213760 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 19:02:37.214861 (+ 1101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:02:37.216707 (+ 1846us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:02:37.216715 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:02:37.217079 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:02:37.218176 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:37.218201 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:37.221831 (+ 3630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:37.221834 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:37.223244 (+ 1410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:37.223260 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:37.224614 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:37.224620 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:37.224813 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:37.229827 (+ 5014us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:02:37.229855 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:02:37.235219 (+ 5364us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:02:37.242385 (+ 7166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:02:37.242395 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:02:37.242415 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:02:37.242857 (+ 442us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:02:37.243396 (+ 539us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:02:37.243399 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:02:37.243402 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:02:37.243601 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:02:37.244541 (+ 940us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:02:37.244548 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:02:37.245118 (+ 570us) client_negotiation.cc:769] Sending connection context
0811 19:02:37.245357 (+ 239us) client_negotiation.cc:240] Negotiation successful
0811 19:02:37.245739 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":125,"threads_started":1}
I20260811 19:02:37.246827 27903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:37.212948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.1:41253 (local address 127.13.109.62:37499)
0811 19:02:37.215396 (+ 2448us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:37.215419 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:37.215468 (+ 49us) server_negotiation.cc:407] Connection header received
0811 19:02:37.215785 (+ 317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:37.215800 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:37.216229 (+ 429us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:37.216588 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:02:37.218977 (+ 2389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:37.221361 (+ 2384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:37.223411 (+ 2050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:37.224177 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:37.224329 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:37.236369 (+ 12040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:02:37.236378 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:02:37.236401 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:02:37.236453 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:02:37.238872 (+ 2419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:37.243039 (+ 4167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:37.243047 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:37.243053 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:37.243253 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:37.243726 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:37.243729 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:37.243731 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:37.244327 (+ 596us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:02:37.244465 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:37.245508 (+ 1043us) server_negotiation.cc:299] Negotiation successful
0811 19:02:37.245818 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2265,"thread_start_us":121,"threads_started":1}
I20260811 19:02:37.250263 27901 heartbeater.cc:347] Connected to a master server at 127.13.109.62:37499
I20260811 19:02:37.250509 27901 heartbeater.cc:464] Registering TS with master...
I20260811 19:02:37.251086 27901 heartbeater.cc:511] Master 127.13.109.62:37499 requested a full tablet report, sending...
W20260811 19:02:37.253614 27901 heartbeater.cc:644] Failed to heartbeat to 127.13.109.62:37499 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 19:02:37.253739 27901 heartbeater.cc:464] Registering TS with master...
I20260811 19:02:37.253957 27901 heartbeater.cc:511] Master 127.13.109.62:37499 requested a full tablet report, sending...
I20260811 19:02:37.254954 27901 heartbeater.cc:464] Registering TS with master...
I20260811 19:02:37.255131 27901 heartbeater.cc:511] Master 127.13.109.62:37499 requested a full tablet report, sending...
W20260811 19:02:37.256016 27901 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 19:02:37.371716 27905 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:02:37.372043 27905 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:02:37.372121 27905 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:02:37.377372 27905 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:02:37.377476 27905 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:02:37.377588 27905 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.2
I20260811 19:02:37.384054 27905 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:32997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.13.109.20:42179/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.2.keytab
--principal=kudu/127.13.109.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.2:39067
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.109.2
--webserver_port=38199
--tserver_master_addrs=127.13.109.62:37499
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27905
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.2
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:02:37.385934 27905 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:02:37.387502 27905 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:02:37.401146 27911 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:02:37.401216 27913 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:02:37.402249 27910 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:02:37.402643 27905 server_base.cc:1034] running on GCE node
I20260811 19:02:37.403322 27905 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:02:37.404213 27905 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:02:37.405380 27905 hybrid_clock.cc:648] HybridClock initialized: now 1786474957405346 us; error 56 us; skew 500 ppm
Aug 11 19:02:37 dist-test-slave-0kls krb5kdc[26552](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474957, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:02:37.408560 27905 init.cc:377] Logged in from keytab as kudu/127.13.109.2@KRBTEST.COM (short username kudu)
I20260811 19:02:37.409648 27905 webserver.cc:460] Webserver started at http://127.13.109.2:38199/ using document root <none> and password file <none>
I20260811 19:02:37.410398 27905 fs_manager.cc:359] Metadata directory not provided
I20260811 19:02:37.410524 27905 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 19:02:37 dist-test-slave-0kls krb5kdc[26552](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474957, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.2@KRBTEST.COM for HTTP/127.13.109.20@KRBTEST.COM
I20260811 19:02:37.440910 27905 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 19:02:37.447553 27922 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:37.449390 27905 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.002s sys 0.000s
I20260811 19:02:37.449548 27905 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ts-1/wal
uuid: "3eb0da02d24c4b6cab01b7a221c5566c"
format_stamp: "Formatted at 2026-08-11 19:02:36 on dist-test-slave-0kls"
server_key: "b8da9910129cf168a0d20d5ec3e37180"
server_key_iv: "3170d394ef4b1d5052b9ef855b705a93"
server_key_version: "kuduclusterkey@0"
I20260811 19:02:37.450369 27905 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:02:37.484961 27905 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:02:37.488775 27905 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:02:37.489017 27905 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:02:37.489918 27905 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:02:37.491022 27905 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:02:37.491092 27905 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:37.491153 27905 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:02:37.491189 27905 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:37.502677 27905 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.2:39067
I20260811 19:02:37.504142 27905 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
I20260811 19:02:37.507452 28049 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.2:39067 every 8 connection(s)
I20260811 19:02:37.510146 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 27905
I20260811 19:02:37.510663 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.109.20:42179/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.3:34487
--local_ip_for_outbound_sockets=127.13.109.3
--tserver_master_addrs=127.13.109.62:37499
--webserver_port=42993
--webserver_interface=127.13.109.3
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.3.keytab
--principal=kudu/127.13.109.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:32997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.13.109.20:42179/kms with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 19:02:37 dist-test-slave-0kls krb5kdc[26552](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474957, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.2@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
I20260811 19:02:37.527652 27903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:37.509551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.2:38281 (local address 127.13.109.62:37499)
0811 19:02:37.509709 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:37.509712 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:37.513256 (+ 3544us) server_negotiation.cc:407] Connection header received
0811 19:02:37.514783 (+ 1527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:37.514789 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:37.514852 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:37.514911 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:02:37.516759 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:37.517705 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:37.519729 (+ 2024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:37.520121 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:37.520240 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:37.523343 (+ 3103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:02:37.523347 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:02:37.523350 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:02:37.523378 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:02:37.525151 (+ 1773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:37.525820 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:37.525824 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:37.525826 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:37.525913 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:37.526316 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:37.526320 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:37.526322 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:37.526564 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:02:37.526677 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:37.527332 (+ 655us) server_negotiation.cc:299] Negotiation successful
0811 19:02:37.527480 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 19:02:37.528644 28053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:37.509941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:37499 (local address 127.13.109.2:38281)
0811 19:02:37.513046 (+ 3105us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:37.513116 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0811 19:02:37.514496 (+ 1380us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:02:37.515355 (+ 859us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:02:37.515367 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:02:37.515760 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:02:37.516538 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:37.516575 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:37.517919 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:37.517925 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:37.519545 (+ 1620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:37.519572 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:37.520323 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:37.520329 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:37.520595 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:37.521266 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:02:37.521294 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:02:37.523134 (+ 1840us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:02:37.525284 (+ 2150us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:02:37.525291 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:02:37.525309 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:02:37.525672 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:02:37.526009 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:02:37.526012 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:02:37.526014 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:02:37.526199 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:02:37.526682 (+ 483us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:02:37.526690 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:02:37.527025 (+ 335us) client_negotiation.cc:769] Sending connection context
0811 19:02:37.527233 (+ 208us) client_negotiation.cc:240] Negotiation successful
0811 19:02:37.527591 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2831,"thread_start_us":161,"threads_started":1}
I20260811 19:02:37.530105 28050 heartbeater.cc:347] Connected to a master server at 127.13.109.62:37499
I20260811 19:02:37.530359 28050 heartbeater.cc:464] Registering TS with master...
I20260811 19:02:37.530984 28050 heartbeater.cc:511] Master 127.13.109.62:37499 requested a full tablet report, sending...
W20260811 19:02:37.532784 28050 heartbeater.cc:644] Failed to heartbeat to 127.13.109.62:37499 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 19:02:37.532912 28050 heartbeater.cc:464] Registering TS with master...
I20260811 19:02:37.533097 28050 heartbeater.cc:511] Master 127.13.109.62:37499 requested a full tablet report, sending...
I20260811 19:02:37.533960 28050 heartbeater.cc:464] Registering TS with master...
I20260811 19:02:37.534135 28050 heartbeater.cc:511] Master 127.13.109.62:37499 requested a full tablet report, sending...
W20260811 19:02:37.534925 28050 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 19:02:37.658454 28054 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:02:37.658757 28054 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:02:37.658804 28054 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:02:37.664592 28054 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:02:37.664703 28054 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:02:37.664801 28054 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.3
I20260811 19:02:37.672240 28054 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:32997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.13.109.20:42179/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.3.keytab
--principal=kudu/127.13.109.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.3:34487
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.109.3
--webserver_port=42993
--tserver_master_addrs=127.13.109.62:37499
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28054
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.3
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:02:37.674019 28054 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:02:37.675510 28054 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:02:37.687611 28064 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:02:37.687693 28062 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:02:37.687824 28061 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:02:37.688616 28054 server_base.cc:1034] running on GCE node
I20260811 19:02:37.689149 28054 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:02:37.689985 28054 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:02:37.691190 28054 hybrid_clock.cc:648] HybridClock initialized: now 1786474957691157 us; error 55 us; skew 500 ppm
Aug 11 19:02:37 dist-test-slave-0kls krb5kdc[26552](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474957, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:02:37.695027 28054 init.cc:377] Logged in from keytab as kudu/127.13.109.3@KRBTEST.COM (short username kudu)
I20260811 19:02:37.696440 28054 webserver.cc:460] Webserver started at http://127.13.109.3:42993/ using document root <none> and password file <none>
I20260811 19:02:37.697238 28054 fs_manager.cc:359] Metadata directory not provided
I20260811 19:02:37.697347 28054 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 19:02:37 dist-test-slave-0kls krb5kdc[26552](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474957, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.3@KRBTEST.COM for HTTP/127.13.109.20@KRBTEST.COM
I20260811 19:02:37.719515 28054 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.002s sys 0.001s
I20260811 19:02:37.726804 28074 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:37.728929 28054 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.002s sys 0.001s
I20260811 19:02:37.729117 28054 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ts-2/wal
uuid: "df2d01a188784505b8dd3d9e81b6a07d"
format_stamp: "Formatted at 2026-08-11 19:02:36 on dist-test-slave-0kls"
server_key: "921be8086db079a6e791f97db77638ad"
server_key_iv: "049b2f99a3765f1e2ea8fd6365f81646"
server_key_version: "kuduclusterkey@0"
I20260811 19:02:37.730041 28054 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:02:37.758231 28054 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:02:37.762283 28054 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:02:37.762508 28054 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:02:37.763278 28054 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:02:37.764389 28054 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:02:37.764470 28054 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:37.764551 28054 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:02:37.764593 28054 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:37.775039 28054 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.3:34487
I20260811 19:02:37.775104 28217 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.3:34487 every 8 connection(s)
I20260811 19:02:37.776258 28054 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
I20260811 19:02:37.779174 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 28054
Aug 11 19:02:37 dist-test-slave-0kls krb5kdc[26552](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474957, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.3@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
I20260811 19:02:37.793449 27903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:37.778599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.3:36527 (local address 127.13.109.62:37499)
0811 19:02:37.778767 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:37.778772 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:37.779936 (+ 1164us) server_negotiation.cc:407] Connection header received
0811 19:02:37.781252 (+ 1316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:37.781255 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:37.781303 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:37.781416 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:02:37.782753 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:37.783644 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:37.785409 (+ 1765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:37.785636 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:37.785786 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:37.788742 (+ 2956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:02:37.788746 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:02:37.788749 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:02:37.788777 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:02:37.790636 (+ 1859us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:37.791360 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:37.791364 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:37.791366 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:37.791462 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:37.791957 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:37.791961 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:37.791962 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:37.792184 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:02:37.792265 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:37.793056 (+ 791us) server_negotiation.cc:299] Negotiation successful
0811 19:02:37.793218 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":31,"server-negotiator.queue_time_us":65}
I20260811 19:02:37.793727 28223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:37.780246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47024 (local address 127.13.109.62:37499)
0811 19:02:37.780524 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:37.780531 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:37.781575 (+ 1044us) server_negotiation.cc:407] Connection header received
0811 19:02:37.781697 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:37.781700 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:37.781742 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:37.781879 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:02:37.782804 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:37.783644 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:37.786630 (+ 2986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:37.786884 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:37.787006 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:37.789416 (+ 2410us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:02:37.789420 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:02:37.789422 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:02:37.789468 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:02:37.791315 (+ 1847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:37.791764 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:37.791769 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:37.791771 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:37.791856 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:37.792494 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:37.792497 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:37.792499 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:37.792678 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:02:37.792898 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:37.793096 (+ 198us) server_negotiation.cc:299] Negotiation successful
0811 19:02:37.793314 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":105,"server-negotiator.queue_time_us":209,"thread_start_us":136,"threads_started":1}
I20260811 19:02:37.795328 28221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:37.779126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:37499 (local address 127.13.109.3:36527)
0811 19:02:37.779755 (+ 629us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:37.779819 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0811 19:02:37.780998 (+ 1179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:02:37.781660 (+ 662us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:02:37.781673 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:02:37.782076 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:02:37.782600 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:37.782616 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:37.783803 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:37.783807 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:37.785251 (+ 1444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:37.785265 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:37.785866 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:37.785871 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:37.786076 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:37.786788 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:02:37.786811 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:02:37.788512 (+ 1701us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:02:37.790781 (+ 2269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:02:37.790790 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:02:37.790812 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:02:37.791232 (+ 420us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:02:37.791589 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:02:37.791592 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:02:37.791594 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:02:37.791838 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:02:37.792300 (+ 462us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:02:37.792309 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:02:37.792731 (+ 422us) client_negotiation.cc:769] Sending connection context
0811 19:02:37.793385 (+ 654us) client_negotiation.cc:240] Negotiation successful
0811 19:02:37.794127 (+ 742us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":170,"threads_started":1}
I20260811 19:02:37.796710 28218 heartbeater.cc:347] Connected to a master server at 127.13.109.62:37499
I20260811 19:02:37.797001 28218 heartbeater.cc:464] Registering TS with master...
I20260811 19:02:37.797585 28218 heartbeater.cc:511] Master 127.13.109.62:37499 requested a full tablet report, sending...
I20260811 19:02:37.799157 27676 ts_manager.cc:194] Registered new tserver with Master: df2d01a188784505b8dd3d9e81b6a07d (127.13.109.3:34487)
I20260811 19:02:37.800643 27676 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.109.3@KRBTEST.COM'} at 127.13.109.3:36527
Aug 11 19:02:38 dist-test-slave-0kls krb5kdc[26552](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786474958, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:02:38.256271 27901 heartbeater.cc:464] Registering TS with master...
I20260811 19:02:38.256695 27901 heartbeater.cc:511] Master 127.13.109.62:37499 requested a full tablet report, sending...
I20260811 19:02:38.257687 27672 ts_manager.cc:194] Registered new tserver with Master: a76ab6148c384490b58109484e6fac3f (127.13.109.1:43647)
I20260811 19:02:38.258370 27672 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.109.1@KRBTEST.COM'} at 127.13.109.1:41253
Aug 11 19:02:38 dist-test-slave-0kls krb5kdc[26552](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474958, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.62@KRBTEST.COM for HTTP/127.13.109.20@KRBTEST.COM
I20260811 19:02:38.535156 28050 heartbeater.cc:464] Registering TS with master...
I20260811 19:02:38.535498 28050 heartbeater.cc:511] Master 127.13.109.62:37499 requested a full tablet report, sending...
I20260811 19:02:38.536432 27672 ts_manager.cc:194] Registered new tserver with Master: 3eb0da02d24c4b6cab01b7a221c5566c (127.13.109.2:39067)
I20260811 19:02:38.537258 27672 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.109.2@KRBTEST.COM'} at 127.13.109.2:38281
I20260811 19:02:38.541456 13748 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 19:02:38.561805 28233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:38.545683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47034 (local address 127.13.109.62:37499)
0811 19:02:38.546205 (+ 522us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:38.546210 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:38.551683 (+ 5473us) server_negotiation.cc:407] Connection header received
0811 19:02:38.551748 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:38.551751 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:38.551811 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:38.551897 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:02:38.552910 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:38.553771 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:38.554919 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:38.555234 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:38.555332 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:38.557345 (+ 2013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:02:38.557349 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:02:38.557352 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:02:38.557384 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:02:38.559315 (+ 1931us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:38.559871 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:38.559875 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:38.559877 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:38.559975 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:38.560300 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:38.560303 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:38.560305 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:38.560552 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:02:38.560706 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:38.561435 (+ 729us) server_negotiation.cc:299] Negotiation successful
0811 19:02:38.561588 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":123,"threads_started":1}
I20260811 19:02:38.566818 27672 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47034:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 19:02:38.571554 27672 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 19:02:38.594539 28240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:38.585155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.1:43647 (local address 127.0.0.1:54216)
0811 19:02:38.586481 (+ 1326us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:38.586539 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 19:02:38.586685 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:02:38.587655 (+ 970us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:02:38.587661 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:02:38.587685 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:02:38.587827 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:38.587833 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:38.590223 (+ 2390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:38.590227 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:38.592602 (+ 2375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:38.592614 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:38.593863 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:38.593867 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:38.593991 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:38.594038 (+ 47us) client_negotiation.cc:769] Sending connection context
0811 19:02:38.594208 (+ 170us) client_negotiation.cc:240] Negotiation successful
0811 19:02:38.594361 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1134,"thread_start_us":132,"threads_started":1}
I20260811 19:02:38.596664 28241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:38.585433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54216 (local address 127.13.109.1:43647)
0811 19:02:38.586986 (+ 1553us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:38.587004 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:38.587028 (+ 24us) server_negotiation.cc:407] Connection header received
0811 19:02:38.587113 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:38.587118 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:38.587319 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:38.587475 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:02:38.587960 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:38.590069 (+ 2109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:38.592750 (+ 2681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:38.593650 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:38.593756 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:38.596319 (+ 2563us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:38.596384 (+ 65us) server_negotiation.cc:299] Negotiation successful
0811 19:02:38.596502 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1443,"thread_start_us":100,"threads_started":1}
I20260811 19:02:38.596900 28242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:38.587589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.3:34487 (local address 127.0.0.1:60580)
0811 19:02:38.588897 (+ 1308us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:38.588908 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 19:02:38.589011 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:02:38.590666 (+ 1655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:02:38.590669 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:02:38.590682 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:02:38.590771 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:38.590776 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:38.592213 (+ 1437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:38.592216 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:38.593438 (+ 1222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:38.593450 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:38.596442 (+ 2992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:38.596446 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:38.596581 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:38.596607 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 19:02:38.596681 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 19:02:38.596751 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1254,"thread_start_us":93,"threads_started":1}
I20260811 19:02:38.597659 28245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:38.588002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60580 (local address 127.13.109.3:34487)
0811 19:02:38.588769 (+ 767us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:38.588775 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:38.589098 (+ 323us) server_negotiation.cc:407] Connection header received
0811 19:02:38.589179 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:38.589185 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:38.589384 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:38.589517 (+ 133us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:02:38.590891 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:38.592068 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:38.593598 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:38.594641 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:38.596209 (+ 1568us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:38.596380 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:38.597359 (+ 979us) server_negotiation.cc:299] Negotiation successful
0811 19:02:38.597487 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":664,"thread_start_us":105,"threads_started":1}
I20260811 19:02:38.600564 27819 tablet_service.cc:1467] Processing CreateTablet for tablet bc27e7c25fbb46d0806e1492d28f4514 (DEFAULT_TABLE table=test-table [id=cad20c3d74d24733b579a23b989a45c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:02:38.602242 27819 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc27e7c25fbb46d0806e1492d28f4514. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:02:38.601320 28143 tablet_service.cc:1467] Processing CreateTablet for tablet bc27e7c25fbb46d0806e1492d28f4514 (DEFAULT_TABLE table=test-table [id=cad20c3d74d24733b579a23b989a45c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:02:38.603688 28143 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc27e7c25fbb46d0806e1492d28f4514. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:02:38.614089 28247 tablet_bootstrap.cc:492] T bc27e7c25fbb46d0806e1492d28f4514 P a76ab6148c384490b58109484e6fac3f: Bootstrap starting.
I20260811 19:02:38.617408 28248 tablet_bootstrap.cc:492] T bc27e7c25fbb46d0806e1492d28f4514 P df2d01a188784505b8dd3d9e81b6a07d: Bootstrap starting.
I20260811 19:02:38.618947 28244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:38.586383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.2:39067 (local address 127.0.0.1:59802)
0811 19:02:38.588413 (+ 2030us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:38.588429 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 19:02:38.588514 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:02:38.590417 (+ 1903us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:02:38.590420 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:02:38.590439 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:02:38.590556 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:38.590563 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:38.609007 (+ 18444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:38.609012 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:38.616869 (+ 7857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:38.616889 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:38.618457 (+ 1568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:38.618461 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:38.618577 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:38.618606 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 19:02:38.618687 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 19:02:38.618786 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1968,"thread_start_us":121,"threads_started":1}
I20260811 19:02:38.619565 28243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:38.586836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59802 (local address 127.13.109.2:39067)
0811 19:02:38.589849 (+ 3013us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:38.589855 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:38.589878 (+ 23us) server_negotiation.cc:407] Connection header received
0811 19:02:38.589952 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:38.589957 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:38.590136 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:38.590267 (+ 131us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:02:38.593876 (+ 3609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:38.596080 (+ 2204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:38.617045 (+ 20965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:38.618026 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:38.618171 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:38.618369 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:38.619273 (+ 904us) server_negotiation.cc:299] Negotiation successful
0811 19:02:38.619405 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2868,"thread_start_us":116,"threads_started":1}
I20260811 19:02:38.616926 28247 tablet_bootstrap.cc:654] T bc27e7c25fbb46d0806e1492d28f4514 P a76ab6148c384490b58109484e6fac3f: Neither blocks nor log segments found. Creating new log.
I20260811 19:02:38.620875 28248 tablet_bootstrap.cc:654] T bc27e7c25fbb46d0806e1492d28f4514 P df2d01a188784505b8dd3d9e81b6a07d: Neither blocks nor log segments found. Creating new log.
I20260811 19:02:38.620853 28247 log.cc:824] T bc27e7c25fbb46d0806e1492d28f4514 P a76ab6148c384490b58109484e6fac3f: Log is configured to *not* fsync() on all Append() calls
I20260811 19:02:38.621776 28248 log.cc:824] T bc27e7c25fbb46d0806e1492d28f4514 P df2d01a188784505b8dd3d9e81b6a07d: Log is configured to *not* fsync() on all Append() calls
I20260811 19:02:38.623097 27976 tablet_service.cc:1467] Processing CreateTablet for tablet bc27e7c25fbb46d0806e1492d28f4514 (DEFAULT_TABLE table=test-table [id=cad20c3d74d24733b579a23b989a45c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:02:38.624461 27976 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc27e7c25fbb46d0806e1492d28f4514. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:02:38.626204 28247 tablet_bootstrap.cc:492] T bc27e7c25fbb46d0806e1492d28f4514 P a76ab6148c384490b58109484e6fac3f: No bootstrap required, opened a new log
I20260811 19:02:38.626504 28247 ts_tablet_manager.cc:1397] T bc27e7c25fbb46d0806e1492d28f4514 P a76ab6148c384490b58109484e6fac3f: Time spent bootstrapping tablet: real 0.013s user 0.007s sys 0.000s
I20260811 19:02:38.630836 28247 raft_consensus.cc:348] T bc27e7c25fbb46d0806e1492d28f4514 P a76ab6148c384490b58109484e6fac3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a76ab6148c384490b58109484e6fac3f" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 43647 } } peers { permanent_uuid: "3eb0da02d24c4b6cab01b7a221c5566c" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 39067 } } peers { permanent_uuid: "df2d01a188784505b8dd3d9e81b6a07d" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 34487 } }
I20260811 19:02:38.631350 28247 raft_consensus.cc:374] T bc27e7c25fbb46d0806e1492d28f4514 P a76ab6148c384490b58109484e6fac3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:02:38.631429 28247 raft_consensus.cc:729] T bc27e7c25fbb46d0806e1492d28f4514 P a76ab6148c384490b58109484e6fac3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a76ab6148c384490b58109484e6fac3f, State: Initialized, Role: FOLLOWER
I20260811 19:02:38.632050 28247 consensus_queue.cc:260] T bc27e7c25fbb46d0806e1492d28f4514 P a76ab6148c384490b58109484e6fac3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a76ab6148c384490b58109484e6fac3f" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 43647 } } peers { permanent_uuid: "3eb0da02d24c4b6cab01b7a221c5566c" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 39067 } } peers { permanent_uuid: "df2d01a188784505b8dd3d9e81b6a07d" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 34487 } }
I20260811 19:02:38.633584 28247 ts_tablet_manager.cc:1428] T bc27e7c25fbb46d0806e1492d28f4514 P a76ab6148c384490b58109484e6fac3f: Time spent starting tablet: real 0.007s user 0.002s sys 0.005s
I20260811 19:02:38.634109 27901 heartbeater.cc:503] Master 127.13.109.62:37499 was elected leader, sending a full tablet report...
I20260811 19:02:38.635671 28253 tablet_bootstrap.cc:492] T bc27e7c25fbb46d0806e1492d28f4514 P 3eb0da02d24c4b6cab01b7a221c5566c: Bootstrap starting.
I20260811 19:02:38.640316 28248 tablet_bootstrap.cc:492] T bc27e7c25fbb46d0806e1492d28f4514 P df2d01a188784505b8dd3d9e81b6a07d: No bootstrap required, opened a new log
I20260811 19:02:38.640579 28248 ts_tablet_manager.cc:1397] T bc27e7c25fbb46d0806e1492d28f4514 P df2d01a188784505b8dd3d9e81b6a07d: Time spent bootstrapping tablet: real 0.023s user 0.006s sys 0.000s
I20260811 19:02:38.641269 28253 tablet_bootstrap.cc:654] T bc27e7c25fbb46d0806e1492d28f4514 P 3eb0da02d24c4b6cab01b7a221c5566c: Neither blocks nor log segments found. Creating new log.
I20260811 19:02:38.642531 28253 log.cc:824] T bc27e7c25fbb46d0806e1492d28f4514 P 3eb0da02d24c4b6cab01b7a221c5566c: Log is configured to *not* fsync() on all Append() calls
I20260811 19:02:38.645503 28253 tablet_bootstrap.cc:492] T bc27e7c25fbb46d0806e1492d28f4514 P 3eb0da02d24c4b6cab01b7a221c5566c: No bootstrap required, opened a new log
I20260811 19:02:38.645732 28253 ts_tablet_manager.cc:1397] T bc27e7c25fbb46d0806e1492d28f4514 P 3eb0da02d24c4b6cab01b7a221c5566c: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260811 19:02:38.645838 28248 raft_consensus.cc:348] T bc27e7c25fbb46d0806e1492d28f4514 P df2d01a188784505b8dd3d9e81b6a07d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a76ab6148c384490b58109484e6fac3f" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 43647 } } peers { permanent_uuid: "3eb0da02d24c4b6cab01b7a221c5566c" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 39067 } } peers { permanent_uuid: "df2d01a188784505b8dd3d9e81b6a07d" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 34487 } }
I20260811 19:02:38.646145 28248 raft_consensus.cc:374] T bc27e7c25fbb46d0806e1492d28f4514 P df2d01a188784505b8dd3d9e81b6a07d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:02:38.646199 28248 raft_consensus.cc:729] T bc27e7c25fbb46d0806e1492d28f4514 P df2d01a188784505b8dd3d9e81b6a07d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df2d01a188784505b8dd3d9e81b6a07d, State: Initialized, Role: FOLLOWER
I20260811 19:02:38.646750 28248 consensus_queue.cc:260] T bc27e7c25fbb46d0806e1492d28f4514 P df2d01a188784505b8dd3d9e81b6a07d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a76ab6148c384490b58109484e6fac3f" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 43647 } } peers { permanent_uuid: "3eb0da02d24c4b6cab01b7a221c5566c" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 39067 } } peers { permanent_uuid: "df2d01a188784505b8dd3d9e81b6a07d" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 34487 } }
I20260811 19:02:38.647944 28248 ts_tablet_manager.cc:1428] T bc27e7c25fbb46d0806e1492d28f4514 P df2d01a188784505b8dd3d9e81b6a07d: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260811 19:02:38.649613 28218 heartbeater.cc:503] Master 127.13.109.62:37499 was elected leader, sending a full tablet report...
I20260811 19:02:38.650059 28253 raft_consensus.cc:348] T bc27e7c25fbb46d0806e1492d28f4514 P 3eb0da02d24c4b6cab01b7a221c5566c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a76ab6148c384490b58109484e6fac3f" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 43647 } } peers { permanent_uuid: "3eb0da02d24c4b6cab01b7a221c5566c" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 39067 } } peers { permanent_uuid: "df2d01a188784505b8dd3d9e81b6a07d" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 34487 } }
I20260811 19:02:38.650484 28253 raft_consensus.cc:374] T bc27e7c25fbb46d0806e1492d28f4514 P 3eb0da02d24c4b6cab01b7a221c5566c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:02:38.650592 28253 raft_consensus.cc:729] T bc27e7c25fbb46d0806e1492d28f4514 P 3eb0da02d24c4b6cab01b7a221c5566c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3eb0da02d24c4b6cab01b7a221c5566c, State: Initialized, Role: FOLLOWER
I20260811 19:02:38.651185 28253 consensus_queue.cc:260] T bc27e7c25fbb46d0806e1492d28f4514 P 3eb0da02d24c4b6cab01b7a221c5566c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a76ab6148c384490b58109484e6fac3f" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 43647 } } peers { permanent_uuid: "3eb0da02d24c4b6cab01b7a221c5566c" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 39067 } } peers { permanent_uuid: "df2d01a188784505b8dd3d9e81b6a07d" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 34487 } }
I20260811 19:02:38.652505 28253 ts_tablet_manager.cc:1428] T bc27e7c25fbb46d0806e1492d28f4514 P 3eb0da02d24c4b6cab01b7a221c5566c: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260811 19:02:38.652902 28050 heartbeater.cc:503] Master 127.13.109.62:37499 was elected leader, sending a full tablet report...
W20260811 19:02:38.712575 27902 tablet.cc:2369] T bc27e7c25fbb46d0806e1492d28f4514 P a76ab6148c384490b58109484e6fac3f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 19:02:38.760099 28051 tablet.cc:2369] T bc27e7c25fbb46d0806e1492d28f4514 P 3eb0da02d24c4b6cab01b7a221c5566c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 19:02:38.778749 28220 tablet.cc:2369] T bc27e7c25fbb46d0806e1492d28f4514 P df2d01a188784505b8dd3d9e81b6a07d: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 11 19:02:38 dist-test-slave-0kls krb5kdc[26552](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474958, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.62@KRBTEST.COM for HTTP/127.13.109.20@KRBTEST.COM
I20260811 19:02:38.860096 28257 raft_consensus.cc:482] T bc27e7c25fbb46d0806e1492d28f4514 P 3eb0da02d24c4b6cab01b7a221c5566c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:02:38.860401 28257 raft_consensus.cc:504] T bc27e7c25fbb46d0806e1492d28f4514 P 3eb0da02d24c4b6cab01b7a221c5566c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a76ab6148c384490b58109484e6fac3f" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 43647 } } peers { permanent_uuid: "3eb0da02d24c4b6cab01b7a221c5566c" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 39067 } } peers { permanent_uuid: "df2d01a188784505b8dd3d9e81b6a07d" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 34487 } }
I20260811 19:02:38.861922 28257 leader_election.cc:288] T bc27e7c25fbb46d0806e1492d28f4514 P 3eb0da02d24c4b6cab01b7a221c5566c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a76ab6148c384490b58109484e6fac3f (127.13.109.1:43647), df2d01a188784505b8dd3d9e81b6a07d (127.13.109.3:34487)
I20260811 19:02:38.869122 28241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:38.863482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.2:43211 (local address 127.13.109.1:43647)
0811 19:02:38.863651 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:38.863657 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:38.864395 (+ 738us) server_negotiation.cc:407] Connection header received
0811 19:02:38.864628 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:38.864632 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:38.864690 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:38.864746 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:02:38.865244 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:38.866072 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:38.867754 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:38.868352 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:38.868452 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:38.868510 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:38.868879 (+ 369us) server_negotiation.cc:299] Negotiation successful
0811 19:02:38.868971 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 19:02:38.869868 28262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:38.862395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.1:43647 (local address 127.13.109.2:43211)
0811 19:02:38.864305 (+ 1910us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:38.864322 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 19:02:38.864487 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:02:38.864913 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:02:38.864916 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:02:38.864938 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:02:38.865096 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:38.865104 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:38.866249 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:38.866253 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:38.867594 (+ 1341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:38.867613 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:38.868614 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:38.868618 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:38.868725 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:38.868751 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 19:02:38.869591 (+ 840us) client_negotiation.cc:240] Negotiation successful
0811 19:02:38.869692 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1740,"thread_start_us":144,"threads_started":1}
I20260811 19:02:38.870750 27845 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc27e7c25fbb46d0806e1492d28f4514" candidate_uuid: "3eb0da02d24c4b6cab01b7a221c5566c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a76ab6148c384490b58109484e6fac3f" is_pre_election: true
I20260811 19:02:38.871110 27845 raft_consensus.cc:2454] T bc27e7c25fbb46d0806e1492d28f4514 P a76ab6148c384490b58109484e6fac3f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3eb0da02d24c4b6cab01b7a221c5566c in term 0.
I20260811 19:02:38.871690 27925 leader_election.cc:302] T bc27e7c25fbb46d0806e1492d28f4514 P 3eb0da02d24c4b6cab01b7a221c5566c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3eb0da02d24c4b6cab01b7a221c5566c, a76ab6148c384490b58109484e6fac3f; no voters:
I20260811 19:02:38.872062 28257 raft_consensus.cc:2781] T bc27e7c25fbb46d0806e1492d28f4514 P 3eb0da02d24c4b6cab01b7a221c5566c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 19:02:38.872148 28257 raft_consensus.cc:482] T bc27e7c25fbb46d0806e1492d28f4514 P 3eb0da02d24c4b6cab01b7a221c5566c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 19:02:38.872192 28257 raft_consensus.cc:3037] T bc27e7c25fbb46d0806e1492d28f4514 P 3eb0da02d24c4b6cab01b7a221c5566c [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:02:38.873765 28257 raft_consensus.cc:504] T bc27e7c25fbb46d0806e1492d28f4514 P 3eb0da02d24c4b6cab01b7a221c5566c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a76ab6148c384490b58109484e6fac3f" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 43647 } } peers { permanent_uuid: "3eb0da02d24c4b6cab01b7a221c5566c" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 39067 } } peers { permanent_uuid: "df2d01a188784505b8dd3d9e81b6a07d" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 34487 } }
I20260811 19:02:38.874231 28257 leader_election.cc:288] T bc27e7c25fbb46d0806e1492d28f4514 P 3eb0da02d24c4b6cab01b7a221c5566c [CANDIDATE]: Term 1 election: Requested vote from peers a76ab6148c384490b58109484e6fac3f (127.13.109.1:43647), df2d01a188784505b8dd3d9e81b6a07d (127.13.109.3:34487)
I20260811 19:02:38.875321 27845 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc27e7c25fbb46d0806e1492d28f4514" candidate_uuid: "3eb0da02d24c4b6cab01b7a221c5566c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a76ab6148c384490b58109484e6fac3f"
I20260811 19:02:38.875478 27845 raft_consensus.cc:3037] T bc27e7c25fbb46d0806e1492d28f4514 P a76ab6148c384490b58109484e6fac3f [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:02:38.876055 28245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:38.863638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.2:56983 (local address 127.13.109.3:34487)
0811 19:02:38.863774 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:38.863778 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:38.866037 (+ 2259us) server_negotiation.cc:407] Connection header received
0811 19:02:38.871933 (+ 5896us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:38.871937 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:38.872004 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:38.872058 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:02:38.872532 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:38.873284 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:38.874807 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:38.875364 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:38.875454 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:38.875505 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:38.875845 (+ 340us) server_negotiation.cc:299] Negotiation successful
0811 19:02:38.875927 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 19:02:38.876461 28261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:38.862897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.3:34487 (local address 127.13.109.2:56983)
0811 19:02:38.865937 (+ 3040us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:38.865952 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 19:02:38.866106 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:02:38.872206 (+ 6100us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:02:38.872209 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:02:38.872229 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:02:38.872392 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:38.872400 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:38.873486 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:38.873490 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:38.874660 (+ 1170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:38.874671 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:38.875597 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:38.875601 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:38.875706 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:38.875730 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 19:02:38.876199 (+ 469us) client_negotiation.cc:240] Negotiation successful
0811 19:02:38.876278 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2974,"thread_start_us":92,"threads_started":1}
I20260811 19:02:38.876994 27845 raft_consensus.cc:2454] T bc27e7c25fbb46d0806e1492d28f4514 P a76ab6148c384490b58109484e6fac3f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3eb0da02d24c4b6cab01b7a221c5566c in term 1.
I20260811 19:02:38.878119 28162 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc27e7c25fbb46d0806e1492d28f4514" candidate_uuid: "3eb0da02d24c4b6cab01b7a221c5566c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df2d01a188784505b8dd3d9e81b6a07d" is_pre_election: true
I20260811 19:02:38.878333 28165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc27e7c25fbb46d0806e1492d28f4514" candidate_uuid: "3eb0da02d24c4b6cab01b7a221c5566c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df2d01a188784505b8dd3d9e81b6a07d"
I20260811 19:02:38.878470 28162 raft_consensus.cc:2454] T bc27e7c25fbb46d0806e1492d28f4514 P df2d01a188784505b8dd3d9e81b6a07d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3eb0da02d24c4b6cab01b7a221c5566c in term 0.
I20260811 19:02:38.880535 27925 leader_election.cc:302] T bc27e7c25fbb46d0806e1492d28f4514 P 3eb0da02d24c4b6cab01b7a221c5566c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3eb0da02d24c4b6cab01b7a221c5566c, a76ab6148c384490b58109484e6fac3f; no voters: df2d01a188784505b8dd3d9e81b6a07d
I20260811 19:02:38.880906 28257 raft_consensus.cc:2781] T bc27e7c25fbb46d0806e1492d28f4514 P 3eb0da02d24c4b6cab01b7a221c5566c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:02:38.881157 28257 raft_consensus.cc:686] T bc27e7c25fbb46d0806e1492d28f4514 P 3eb0da02d24c4b6cab01b7a221c5566c [term 1 LEADER]: Becoming Leader. State: Replica: 3eb0da02d24c4b6cab01b7a221c5566c, State: Running, Role: LEADER
I20260811 19:02:38.881536 28257 consensus_queue.cc:237] T bc27e7c25fbb46d0806e1492d28f4514 P 3eb0da02d24c4b6cab01b7a221c5566c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a76ab6148c384490b58109484e6fac3f" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 43647 } } peers { permanent_uuid: "3eb0da02d24c4b6cab01b7a221c5566c" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 39067 } } peers { permanent_uuid: "df2d01a188784505b8dd3d9e81b6a07d" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 34487 } }
I20260811 19:02:38.885507 27676 catalog_manager.cc:5495] T bc27e7c25fbb46d0806e1492d28f4514 P 3eb0da02d24c4b6cab01b7a221c5566c reported cstate change: term changed from 0 to 1, leader changed from <none> to 3eb0da02d24c4b6cab01b7a221c5566c (127.13.109.2). New cstate: current_term: 1 leader_uuid: "3eb0da02d24c4b6cab01b7a221c5566c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a76ab6148c384490b58109484e6fac3f" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 43647 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3eb0da02d24c4b6cab01b7a221c5566c" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 39067 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "df2d01a188784505b8dd3d9e81b6a07d" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 34487 } health_report { overall_health: UNKNOWN } } }
I20260811 19:02:38.962482 28243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:38.957118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59814 (local address 127.13.109.2:39067)
0811 19:02:38.957283 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:38.957288 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:38.957302 (+ 14us) server_negotiation.cc:407] Connection header received
0811 19:02:38.957395 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:38.957397 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:38.957463 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:38.957567 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:02:38.957993 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:38.958739 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:38.960028 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:38.960415 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:38.960541 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:38.961042 (+ 501us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:02:38.961648 (+ 606us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:02:38.961762 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:38.962245 (+ 483us) server_negotiation.cc:299] Negotiation successful
0811 19:02:38.962313 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 19:02:38.970140 28162 raft_consensus.cc:3037] T bc27e7c25fbb46d0806e1492d28f4514 P df2d01a188784505b8dd3d9e81b6a07d [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:02:38.971755 28162 raft_consensus.cc:1261] T bc27e7c25fbb46d0806e1492d28f4514 P df2d01a188784505b8dd3d9e81b6a07d [term 1 FOLLOWER]: Refusing update from remote peer 3eb0da02d24c4b6cab01b7a221c5566c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:02:38.972695 28263 consensus_queue.cc:1035] T bc27e7c25fbb46d0806e1492d28f4514 P 3eb0da02d24c4b6cab01b7a221c5566c [LEADER]: Connected to new peer: Peer: permanent_uuid: "df2d01a188784505b8dd3d9e81b6a07d" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 34487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:02:38.981982 27845 raft_consensus.cc:1261] T bc27e7c25fbb46d0806e1492d28f4514 P a76ab6148c384490b58109484e6fac3f [term 1 FOLLOWER]: Refusing update from remote peer 3eb0da02d24c4b6cab01b7a221c5566c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:02:38.982905 28263 consensus_queue.cc:1035] T bc27e7c25fbb46d0806e1492d28f4514 P 3eb0da02d24c4b6cab01b7a221c5566c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a76ab6148c384490b58109484e6fac3f" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 43647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:02:39.001015 28266 mvcc.cc:205] Tried to move back new op lower bound from 7317401431932612608 to 7317401431582892032. Current Snapshot: MvccSnapshot[applied={T|T < 7317401431932612608}]
I20260811 19:02:39.004540 28269 mvcc.cc:205] Tried to move back new op lower bound from 7317401431932612608 to 7317401431582892032. Current Snapshot: MvccSnapshot[applied={T|T < 7317401431932612608}]
I20260811 19:02:39.015936 28267 mvcc.cc:205] Tried to move back new op lower bound from 7317401431932612608 to 7317401431582892032. Current Snapshot: MvccSnapshot[applied={T|T < 7317401431932612608}]
I20260811 19:02:39.026467 27676 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47034:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:02:39.026692 27676 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47034:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:02:39.030364 27676 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 368bad7350ee4cf69a7a7ec456f9a088: Sending DeleteTablet for 3 replicas of tablet bc27e7c25fbb46d0806e1492d28f4514
I20260811 19:02:39.031901 27819 tablet_service.cc:1514] Processing DeleteTablet for tablet bc27e7c25fbb46d0806e1492d28f4514 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:02:39 UTC) from {username='kudu', principal='kudu/127.13.109.62@KRBTEST.COM'} at 127.0.0.1:54216
I20260811 19:02:39.033102 27976 tablet_service.cc:1514] Processing DeleteTablet for tablet bc27e7c25fbb46d0806e1492d28f4514 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:02:39 UTC) from {username='kudu', principal='kudu/127.13.109.62@KRBTEST.COM'} at 127.0.0.1:59802
I20260811 19:02:39.031901 28143 tablet_service.cc:1514] Processing DeleteTablet for tablet bc27e7c25fbb46d0806e1492d28f4514 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:02:39 UTC) from {username='kudu', principal='kudu/127.13.109.62@KRBTEST.COM'} at 127.0.0.1:60580
I20260811 19:02:39.034534 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 27720
I20260811 19:02:39.042074 28276 tablet_replica.cc:330] T bc27e7c25fbb46d0806e1492d28f4514 P 3eb0da02d24c4b6cab01b7a221c5566c: stopping tablet replica
I20260811 19:02:39.042426 28277 tablet_replica.cc:330] T bc27e7c25fbb46d0806e1492d28f4514 P df2d01a188784505b8dd3d9e81b6a07d: stopping tablet replica
I20260811 19:02:39.042930 28276 raft_consensus.cc:2229] T bc27e7c25fbb46d0806e1492d28f4514 P 3eb0da02d24c4b6cab01b7a221c5566c [term 1 LEADER]: Raft consensus shutting down.
I20260811 19:02:39.043319 28277 raft_consensus.cc:2229] T bc27e7c25fbb46d0806e1492d28f4514 P df2d01a188784505b8dd3d9e81b6a07d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:02:39.043713 28277 raft_consensus.cc:2258] T bc27e7c25fbb46d0806e1492d28f4514 P df2d01a188784505b8dd3d9e81b6a07d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:02:39.045723 28276 raft_consensus.cc:2258] T bc27e7c25fbb46d0806e1492d28f4514 P 3eb0da02d24c4b6cab01b7a221c5566c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:02:39.052608 28277 ts_tablet_manager.cc:1905] T bc27e7c25fbb46d0806e1492d28f4514 P df2d01a188784505b8dd3d9e81b6a07d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:02:39.054029 28276 ts_tablet_manager.cc:1905] T bc27e7c25fbb46d0806e1492d28f4514 P 3eb0da02d24c4b6cab01b7a221c5566c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:02:39.059424 28277 ts_tablet_manager.cc:1918] T bc27e7c25fbb46d0806e1492d28f4514 P df2d01a188784505b8dd3d9e81b6a07d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:02:39.059674 28277 log.cc:1196] T bc27e7c25fbb46d0806e1492d28f4514 P df2d01a188784505b8dd3d9e81b6a07d: Deleting WAL directory at /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ts-2/wal/wals/bc27e7c25fbb46d0806e1492d28f4514
I20260811 19:02:39.060200 28277 ts_tablet_manager.cc:1939] T bc27e7c25fbb46d0806e1492d28f4514 P df2d01a188784505b8dd3d9e81b6a07d: Deleting consensus metadata
I20260811 19:02:39.061887 27661 catalog_manager.cc:4841] TS df2d01a188784505b8dd3d9e81b6a07d (127.13.109.3:34487): tablet bc27e7c25fbb46d0806e1492d28f4514 (table test-table [id=cad20c3d74d24733b579a23b989a45c3]) successfully deleted
I20260811 19:02:39.062536 28276 ts_tablet_manager.cc:1918] T bc27e7c25fbb46d0806e1492d28f4514 P 3eb0da02d24c4b6cab01b7a221c5566c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:02:39.062706 28276 log.cc:1196] T bc27e7c25fbb46d0806e1492d28f4514 P 3eb0da02d24c4b6cab01b7a221c5566c: Deleting WAL directory at /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474792496492-13748-0/minicluster-data/ts-1/wal/wals/bc27e7c25fbb46d0806e1492d28f4514
I20260811 19:02:39.063081 28276 ts_tablet_manager.cc:1939] T bc27e7c25fbb46d0806e1492d28f4514 P 3eb0da02d24c4b6cab01b7a221c5566c: Deleting consensus metadata
I20260811 19:02:39.064460 27660 catalog_manager.cc:4841] TS 3eb0da02d24c4b6cab01b7a221c5566c (127.13.109.2:39067): tablet bc27e7c25fbb46d0806e1492d28f4514 (table test-table [id=cad20c3d74d24733b579a23b989a45c3]) successfully deleted
W20260811 19:02:39.064977 27662 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.109.1:43647 (error 108)
I20260811 19:02:39.065203 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 27905
I20260811 19:02:39.072816 28244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:39.072307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.1:43647 (local address 127.0.0.1:54232)
0811 19:02:39.072596 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:39.072693 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.1:43647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154}
W20260811 19:02:39.073071 27662 catalog_manager.cc:4584] TS a76ab6148c384490b58109484e6fac3f (127.13.109.1:43647): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bc27e7c25fbb46d0806e1492d28f4514: Network error: Client connection negotiation failed: client connection to 127.13.109.1:43647: connect: Connection refused (error 111)
I20260811 19:02:39.079581 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 28054
I20260811 19:02:39.093530 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 27642
I20260811 19:02:39.105969 13748 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260811 19:02:39.451630 13748 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260811 19:02:39.451758 13748 mini_ranger.cc:66] Stopping Ranger...
I20260811 19:02:39.798444 13748 mini_ranger.cc:68] Stopped Ranger
2026-08-11 19:02:39.798 UTC [26587] LOG: received smart shutdown request
2026-08-11 19:02:39.803 UTC [26587] LOG: background worker "logical replication launcher" (PID 26597) exited with exit code 1
2026-08-11 19:02:39.803 UTC [26592] LOG: shutting down
2026-08-11 19:02:39.994 UTC [26587] LOG: database system is shut down
2026-08-11T19:02:39Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (51129 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474792496492-13748-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:02:40 dist-test-slave-0kls krb5kdc[28288](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:02:40 dist-test-slave-0kls krb5kdc[28288](info): set up 2 sockets
Aug 11 19:02:40 dist-test-slave-0kls krb5kdc[28288](info): commencing operation
krb5kdc: starting...
W20260811 19:02:41.218940 13748 mini_kdc.cc:121] Time spent starting KDC: real 1.076s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474792496492-13748-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474792496492-13748-0/krb5kdc/test-user.keytab.
Aug 11 19:02:41 dist-test-slave-0kls krb5kdc[28288](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474961, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:02:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:02:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.109.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.109.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.109.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab.
Entry for principal kudu/127.13.109.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab.
I20260811 19:02:41.359162 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474792496492-13748-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474792496492-13748-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474792496492-13748-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.62:40127
--webserver_interface=127.13.109.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab
--principal=kudu/127.13.109.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:38471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.109.62:40127,127.13.109.61:33979,127.13.109.60:46713
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:02:41.479223 28301 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:02:41.479619 28301 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:02:41.479712 28301 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:02:41.484140 28301 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:02:41.484273 28301 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:02:41.484309 28301 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:02:41.484355 28301 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:02:41.484414 28301 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:02:41.490509 28301 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:38471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474792496492-13748-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474792496492-13748-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.109.62:40127,127.13.109.61:33979,127.13.109.60:46713
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab
--principal=kudu/127.13.109.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.62:40127
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.109.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28301
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474792496492-13748-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:02:41.491863 28301 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:02:41.493032 28301 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:02:41.502543 28306 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:02:41.502559 28307 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:02:41.502543 28309 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:02:41.502873 28301 server_base.cc:1034] running on GCE node
I20260811 19:02:41.503691 28301 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:02:41.505014 28301 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:02:41.506239 28301 hybrid_clock.cc:648] HybridClock initialized: now 1786474961506220 us; error 30 us; skew 500 ppm
Aug 11 19:02:41 dist-test-slave-0kls krb5kdc[28288](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474961, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:02:41.509246 28301 init.cc:377] Logged in from keytab as kudu/127.13.109.62@KRBTEST.COM (short username kudu)
I20260811 19:02:41.510389 28301 webserver.cc:460] Webserver started at http://127.13.109.62:37875/ using document root <none> and password file <none>
I20260811 19:02:41.511016 28301 fs_manager.cc:359] Metadata directory not provided
I20260811 19:02:41.511128 28301 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:02:41.511394 28301 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:02:41.513860 28301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474792496492-13748-0/minicluster-data/master-0/data/instance:
uuid: "fce34686bc2a445c96c5226ba46b4c86"
format_stamp: "Formatted at 2026-08-11 19:02:41 on dist-test-slave-0kls"
server_key: "733538645ffa44a68065999093b0d4a3"
server_key_iv: "e992128f4dcd1d55e2d755d47808813a"
server_key_version: "encryptionkey@0"
I20260811 19:02:41.514525 28301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474792496492-13748-0/minicluster-data/master-0/wal/instance:
uuid: "fce34686bc2a445c96c5226ba46b4c86"
format_stamp: "Formatted at 2026-08-11 19:02:41 on dist-test-slave-0kls"
server_key: "733538645ffa44a68065999093b0d4a3"
server_key_iv: "e992128f4dcd1d55e2d755d47808813a"
server_key_version: "encryptionkey@0"
I20260811 19:02:41.518879 28301 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 19:02:41.521999 28317 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:41.523379 28301 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 19:02:41.523548 28301 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474792496492-13748-0/minicluster-data/master-0/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474792496492-13748-0/minicluster-data/master-0/wal
uuid: "fce34686bc2a445c96c5226ba46b4c86"
format_stamp: "Formatted at 2026-08-11 19:02:41 on dist-test-slave-0kls"
server_key: "733538645ffa44a68065999093b0d4a3"
server_key_iv: "e992128f4dcd1d55e2d755d47808813a"
server_key_version: "encryptionkey@0"
I20260811 19:02:41.523693 28301 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474792496492-13748-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474792496492-13748-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474792496492-13748-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:02:41.536036 28301 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:02:41.539227 28301 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:02:41.539458 28301 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:02:41.547508 28301 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.62:40127
I20260811 19:02:41.547520 28373 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.62:40127 every 8 connection(s)
I20260811 19:02:41.548530 28301 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
I20260811 19:02:41.551522 28374 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:02:41.555739 28374 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.109.62" port: 40127 } has no permanent_uuid. Determining permanent_uuid...
I20260811 19:02:41.557390 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 28301
I20260811 19:02:41.557530 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474792496492-13748-0/minicluster-data/master-0/wal/instance
I20260811 19:02:41.557956 13748 external_mini_cluster.cc:1428] Setting key 591f124e75d06e8caa4fb3bab99afe89
Aug 11 19:02:41 dist-test-slave-0kls krb5kdc[28288](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474961, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.62@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.109.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.109.61@KRBTEST.COM" created.
I20260811 19:02:41.572902 28377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:41.558371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35492 (local address 127.13.109.62:40127)
0811 19:02:41.558857 (+ 486us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:41.558873 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:41.559237 (+ 364us) server_negotiation.cc:407] Connection header received
0811 19:02:41.560312 (+ 1075us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:41.560316 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:41.560495 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:41.560687 (+ 192us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:02:41.561570 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:41.563013 (+ 1443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:41.564159 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:41.564584 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:41.564709 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:41.567596 (+ 2887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:02:41.567607 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:02:41.567624 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:02:41.567656 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:02:41.569501 (+ 1845us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:41.570151 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:41.570156 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:41.570157 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:41.570266 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:41.570593 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:41.570598 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:41.570600 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:41.571027 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:02:41.571214 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:41.571775 (+ 561us) server_negotiation.cc:299] Negotiation successful
0811 19:02:41.572050 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":48,"server-negotiator.queue_time_us":331,"thread_start_us":159,"threads_started":1}
I20260811 19:02:41.572913 28379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:41.558372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:40127 (local address 127.0.0.1:35492)
0811 19:02:41.559077 (+ 705us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:41.559114 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 19:02:41.559985 (+ 871us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:02:41.560695 (+ 710us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:02:41.560700 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:02:41.561152 (+ 452us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:02:41.561412 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:41.561426 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:41.563181 (+ 1755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:41.563185 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:41.564017 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:41.564028 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:41.564786 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:41.564794 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:41.564967 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:41.565614 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:02:41.565641 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:02:41.567412 (+ 1771us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:02:41.569638 (+ 2226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:02:41.569648 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:02:41.569657 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:02:41.570001 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:02:41.570355 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:02:41.570357 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:02:41.570359 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:02:41.570488 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:02:41.571202 (+ 714us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:02:41.571208 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:02:41.571429 (+ 221us) client_negotiation.cc:769] Sending connection context
0811 19:02:41.571645 (+ 216us) client_negotiation.cc:240] Negotiation successful
0811 19:02:41.572157 (+ 512us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":552,"mutex_wait_us":118,"spinlock_wait_cycles":23936,"thread_start_us":183,"threads_started":1}
I20260811 19:02:41.575696 28374 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.109.61" port: 33979 } has no permanent_uuid. Determining permanent_uuid...
I20260811 19:02:41.576501 28379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:41.576238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.61:33979 (local address 127.0.0.1:58334)
0811 19:02:41.576365 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:41.576438 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.61:33979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260811 19:02:41.576718 28318 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.109.61:33979: connect: Connection refused (error 111)
I20260811 19:02:41.577658 28379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:41.577427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.61:33979 (local address 127.0.0.1:58342)
0811 19:02:41.577550 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:41.577598 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.61:33979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260811 19:02:41.577848 28374 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.109.61:33979: Network error: Client connection negotiation failed: client connection to 127.13.109.61:33979: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.109.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.61.keytab.
Entry for principal kudu/127.13.109.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.61.keytab.
I20260811 19:02:41.589257 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474792496492-13748-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474792496492-13748-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474792496492-13748-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474792496492-13748-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.61:33979
--webserver_interface=127.13.109.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.61.keytab
--principal=kudu/127.13.109.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:38471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.109.62:40127,127.13.109.61:33979,127.13.109.60:46713
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 19:02:41.627075 28374 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.109.61" port: 33979 } attempt: 1
I20260811 19:02:41.628144 28379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:41.627710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.61:33979 (local address 127.0.0.1:58348)
0811 19:02:41.627997 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:41.628058 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.61:33979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195}
I20260811 19:02:41.629029 28379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:41.628763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.61:33979 (local address 127.0.0.1:58350)
0811 19:02:41.628893 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:41.628962 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.61:33979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260811 19:02:41.629158 28374 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.109.61:33979: Network error: Client connection negotiation failed: client connection to 127.13.109.61:33979: connect: Connection refused (error 111)
I20260811 19:02:41.697365 28374 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.109.61" port: 33979 } attempt: 2
I20260811 19:02:41.698412 28379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:41.698030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.61:33979 (local address 127.0.0.1:58364)
0811 19:02:41.698229 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:41.698311 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.61:33979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260811 19:02:41.699276 28379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:41.698998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.61:33979 (local address 127.0.0.1:58380)
0811 19:02:41.699122 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:41.699200 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.61:33979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 19:02:41.699426 28374 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.109.61:33979: Network error: Client connection negotiation failed: client connection to 127.13.109.61:33979: connect: Connection refused (error 111)
W20260811 19:02:41.712451 28385 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:02:41.712770 28385 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:02:41.712862 28385 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:02:41.717306 28385 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:02:41.717386 28385 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:02:41.717406 28385 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:02:41.717425 28385 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:02:41.717442 28385 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:02:41.723766 28385 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:38471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474792496492-13748-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474792496492-13748-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.13.109.62:40127,127.13.109.61:33979,127.13.109.60:46713
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.61.keytab
--principal=kudu/127.13.109.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.61:33979
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474792496492-13748-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.13.109.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28385
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474792496492-13748-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:02:41.725088 28385 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:02:41.726277 28385 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:02:41.736166 28393 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:02:41.736217 28395 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:02:41.736276 28385 server_base.cc:1034] running on GCE node
W20260811 19:02:41.736207 28392 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:02:41.737066 28385 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:02:41.738373 28385 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:02:41.739575 28385 hybrid_clock.cc:648] HybridClock initialized: now 1786474961739558 us; error 31 us; skew 500 ppm
Aug 11 19:02:41 dist-test-slave-0kls krb5kdc[28288](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474961, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:02:41.742378 28385 init.cc:377] Logged in from keytab as kudu/127.13.109.61@KRBTEST.COM (short username kudu)
I20260811 19:02:41.743388 28385 webserver.cc:460] Webserver started at http://127.13.109.61:43245/ using document root <none> and password file <none>
I20260811 19:02:41.744000 28385 fs_manager.cc:359] Metadata directory not provided
I20260811 19:02:41.744095 28385 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:02:41.744335 28385 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:02:41.746723 28385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474792496492-13748-0/minicluster-data/master-1/data/instance:
uuid: "2399f3177b5f4f6aa611a35f49ca5301"
format_stamp: "Formatted at 2026-08-11 19:02:41 on dist-test-slave-0kls"
server_key: "4758bb43c8155bc7aa35b8e3605b5ed1"
server_key_iv: "80b408d840c19cc392e32ec939c655ee"
server_key_version: "encryptionkey@0"
I20260811 19:02:41.747484 28385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474792496492-13748-0/minicluster-data/master-1/wal/instance:
uuid: "2399f3177b5f4f6aa611a35f49ca5301"
format_stamp: "Formatted at 2026-08-11 19:02:41 on dist-test-slave-0kls"
server_key: "4758bb43c8155bc7aa35b8e3605b5ed1"
server_key_iv: "80b408d840c19cc392e32ec939c655ee"
server_key_version: "encryptionkey@0"
I20260811 19:02:41.751317 28385 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 19:02:41.753878 28404 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:41.755138 28385 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 19:02:41.755292 28385 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474792496492-13748-0/minicluster-data/master-1/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474792496492-13748-0/minicluster-data/master-1/wal
uuid: "2399f3177b5f4f6aa611a35f49ca5301"
format_stamp: "Formatted at 2026-08-11 19:02:41 on dist-test-slave-0kls"
server_key: "4758bb43c8155bc7aa35b8e3605b5ed1"
server_key_iv: "80b408d840c19cc392e32ec939c655ee"
server_key_version: "encryptionkey@0"
I20260811 19:02:41.755411 28385 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474792496492-13748-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474792496492-13748-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474792496492-13748-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:02:41.778291 28385 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:02:41.781462 28385 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:02:41.781709 28385 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:02:41.789395 28385 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.61:33979
I20260811 19:02:41.789422 28462 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.61:33979 every 8 connection(s)
I20260811 19:02:41.790486 28385 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474792496492-13748-0/minicluster-data/master-1/data/info.pb
I20260811 19:02:41.790652 28374 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.109.61" port: 33979 } attempt: 3
I20260811 19:02:41.793708 28463 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:02:41.797991 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 28385
I20260811 19:02:41.798147 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474792496492-13748-0/minicluster-data/master-1/wal/instance
I20260811 19:02:41.798549 13748 external_mini_cluster.cc:1428] Setting key 6d729169e23f71ed801f92c94a7174fb
I20260811 19:02:41.798532 28463 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.109.62" port: 40127 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 19:02:41 dist-test-slave-0kls krb5kdc[28288](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474961, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.62@KRBTEST.COM for kudu/127.13.109.61@KRBTEST.COM
I20260811 19:02:41.804399 28379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:41.791255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.61:33979 (local address 127.0.0.1:58394)
0811 19:02:41.791405 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:41.791423 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 19:02:41.791516 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:02:41.793887 (+ 2371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:02:41.793891 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:02:41.794094 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:02:41.794253 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:41.794262 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:41.795511 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:41.795515 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:41.796343 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:41.796355 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:41.797064 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:41.797068 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:41.797185 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:41.797708 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:02:41.797730 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:02:41.799395 (+ 1665us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:02:41.802512 (+ 3117us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:02:41.802516 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:02:41.802518 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:02:41.802732 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:02:41.803088 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:02:41.803093 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:02:41.803095 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:02:41.803192 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:02:41.803823 (+ 631us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:02:41.803826 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:02:41.803944 (+ 118us) client_negotiation.cc:769] Sending connection context
0811 19:02:41.804057 (+ 113us) client_negotiation.cc:240] Negotiation successful
0811 19:02:41.804223 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260811 19:02:41.805459 28465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:41.792024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58394 (local address 127.13.109.61:33979)
0811 19:02:41.792475 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:41.792490 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:41.792528 (+ 38us) server_negotiation.cc:407] Connection header received
0811 19:02:41.793191 (+ 663us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:41.793208 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:41.793564 (+ 356us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:41.793940 (+ 376us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:02:41.794385 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:41.795362 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:41.796470 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:41.796927 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:41.797046 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:41.799627 (+ 2581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:02:41.799634 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:02:41.799649 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:02:41.799692 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:02:41.802390 (+ 2698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:41.802856 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:41.802862 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:41.802875 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:41.802988 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:41.803288 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:41.803292 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:41.803294 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:41.803709 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:02:41.803857 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:41.804212 (+ 355us) server_negotiation.cc:299] Negotiation successful
0811 19:02:41.804545 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":140,"threads_started":1}
I20260811 19:02:41.807567 28374 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.109.60" port: 46713 } has no permanent_uuid. Determining permanent_uuid...
I20260811 19:02:41.808454 28379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:41.808184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.60:46713 (local address 127.0.0.1:56602)
0811 19:02:41.808299 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:41.808358 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.60:46713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260811 19:02:41.809355 28379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:41.809001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.60:46713 (local address 127.0.0.1:56608)
0811 19:02:41.809196 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:41.809261 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.60:46713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
W20260811 19:02:41.809603 28374 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.109.60:46713: Network error: Client connection negotiation failed: client connection to 127.13.109.60:46713: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.13.109.60@KRBTEST.COM; defaulting to no policy
Aug 11 19:02:41 dist-test-slave-0kls krb5kdc[28288](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474961, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.61@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.109.60@KRBTEST.COM" created.
I20260811 19:02:41.814121 28377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:41.800650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35496 (local address 127.13.109.62:40127)
0811 19:02:41.800805 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:41.800810 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:41.801288 (+ 478us) server_negotiation.cc:407] Connection header received
0811 19:02:41.801454 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:41.801458 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:41.801521 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:41.801621 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:02:41.804305 (+ 2684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:41.805221 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:41.806679 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:41.807627 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:41.807758 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:41.810611 (+ 2853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:02:41.810614 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:02:41.810616 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:02:41.810639 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:02:41.812190 (+ 1551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:41.812688 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:41.812691 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:41.812692 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:41.812767 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:41.813078 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:41.813081 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:41.813083 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:41.813322 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:02:41.813433 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:41.813848 (+ 415us) server_negotiation.cc:299] Negotiation successful
0811 19:02:41.813969 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 19:02:41.814141 28468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:41.800577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:40127 (local address 127.0.0.1:35496)
0811 19:02:41.801112 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:41.801186 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0811 19:02:41.801332 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:02:41.801680 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:02:41.801687 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:02:41.802260 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:02:41.802448 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:41.802458 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:41.805396 (+ 2938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:41.805403 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:41.806539 (+ 1136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:41.806552 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:41.807786 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:41.807790 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:41.807929 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:41.808736 (+ 807us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:02:41.808748 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:02:41.810471 (+ 1723us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:02:41.812321 (+ 1850us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:02:41.812358 (+ 37us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:02:41.812363 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:02:41.812588 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:02:41.812848 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:02:41.812850 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:02:41.812851 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:02:41.812964 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:02:41.813445 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:02:41.813455 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:02:41.813581 (+ 126us) client_negotiation.cc:769] Sending connection context
0811 19:02:41.813727 (+ 146us) client_negotiation.cc:240] Negotiation successful
0811 19:02:41.813970 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":128,"threads_started":1}
I20260811 19:02:41.815470 28463 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.109.61" port: 33979 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 19:02:41 dist-test-slave-0kls krb5kdc[28288](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474961, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.61@KRBTEST.COM for kudu/127.13.109.61@KRBTEST.COM
I20260811 19:02:41.823678 28465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:41.816077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58396 (local address 127.13.109.61:33979)
0811 19:02:41.816184 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:41.816187 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:41.816297 (+ 110us) server_negotiation.cc:407] Connection header received
0811 19:02:41.816434 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:41.816437 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:41.816481 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:41.816560 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:02:41.816938 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:41.817448 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:41.818248 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:41.818491 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:41.818600 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:41.820539 (+ 1939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:02:41.820542 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:02:41.820544 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:02:41.820564 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:02:41.822047 (+ 1483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:41.822459 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:41.822464 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:41.822465 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:41.822539 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:41.822808 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:41.822813 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:41.822815 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:41.822994 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:02:41.823105 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:41.823386 (+ 281us) server_negotiation.cc:299] Negotiation successful
0811 19:02:41.823547 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":63,"server-negotiator.queue_time_us":36}
I20260811 19:02:41.823678 28468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:41.815973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.61:33979 (local address 127.0.0.1:58396)
0811 19:02:41.816227 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:41.816237 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 19:02:41.816335 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:02:41.816571 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:02:41.816574 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:02:41.816744 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:02:41.816829 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:41.816834 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:41.817555 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:41.817560 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:41.818136 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:41.818145 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:41.818600 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:41.818603 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:41.818688 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:41.819206 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:02:41.819217 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:02:41.820402 (+ 1185us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:02:41.822174 (+ 1772us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:02:41.822177 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:02:41.822180 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:02:41.822363 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:02:41.822636 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:02:41.822639 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:02:41.822641 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:02:41.822721 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:02:41.823124 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:02:41.823136 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:02:41.823223 (+ 87us) client_negotiation.cc:769] Sending connection context
0811 19:02:41.823342 (+ 119us) client_negotiation.cc:240] Negotiation successful
0811 19:02:41.823488 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221}
I20260811 19:02:41.824843 28463 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.109.60" port: 46713 } has no permanent_uuid. Determining permanent_uuid...
I20260811 19:02:41.825649 28468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:41.825360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.60:46713 (local address 127.0.0.1:56616)
0811 19:02:41.825502 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:41.825582 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.60:46713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260811 19:02:41.826335 28405 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.109.60:46713: connect: Connection refused (error 111)
I20260811 19:02:41.827438 28468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:41.827169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.60:46713 (local address 127.0.0.1:56630)
0811 19:02:41.827274 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:41.827371 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.60:46713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260811 19:02:41.827649 28463 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.109.60:46713: Network error: Client connection negotiation failed: client connection to 127.13.109.60:46713: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.109.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.60.keytab.
Entry for principal kudu/127.13.109.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.60.keytab.
I20260811 19:02:41.830976 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474792496492-13748-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474792496492-13748-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474792496492-13748-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474792496492-13748-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.60:46713
--webserver_interface=127.13.109.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.60.keytab
--principal=kudu/127.13.109.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:38471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.109.62:40127,127.13.109.61:33979,127.13.109.60:46713
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 19:02:41.840770 28374 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.109.60" port: 46713 } attempt: 1
I20260811 19:02:41.841622 28379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:41.841329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.60:46713 (local address 127.0.0.1:56632)
0811 19:02:41.841475 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:41.841542 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.60:46713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260811 19:02:41.842403 28379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:41.842184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.60:46713 (local address 127.0.0.1:56646)
0811 19:02:41.842296 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:41.842349 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.60:46713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260811 19:02:41.842588 28374 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.109.60:46713: Network error: Client connection negotiation failed: client connection to 127.13.109.60:46713: connect: Connection refused (error 111)
I20260811 19:02:41.876942 28463 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.109.60" port: 46713 } attempt: 1
I20260811 19:02:41.877974 28468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:41.877578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.60:46713 (local address 127.0.0.1:56650)
0811 19:02:41.877768 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:41.877871 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.60:46713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260811 19:02:41.878829 28468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:41.878562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.60:46713 (local address 127.0.0.1:56654)
0811 19:02:41.878697 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:41.878760 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.60:46713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260811 19:02:41.878983 28463 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.109.60:46713: Network error: Client connection negotiation failed: client connection to 127.13.109.60:46713: connect: Connection refused (error 111)
I20260811 19:02:41.917766 28374 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.109.60" port: 46713 } attempt: 2
I20260811 19:02:41.918920 28379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:41.918594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.60:46713 (local address 127.0.0.1:56658)
0811 19:02:41.918768 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:41.918836 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.60:46713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260811 19:02:41.919761 28379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:41.919513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.60:46713 (local address 127.0.0.1:56662)
0811 19:02:41.919627 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:41.919685 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.60:46713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260811 19:02:41.919902 28374 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.109.60:46713: Network error: Client connection negotiation failed: client connection to 127.13.109.60:46713: connect: Connection refused (error 111)
I20260811 19:02:41.947173 28463 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.109.60" port: 46713 } attempt: 2
I20260811 19:02:41.948088 28468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:41.947762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.60:46713 (local address 127.0.0.1:56672)
0811 19:02:41.947925 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:41.948004 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.60:46713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260811 19:02:41.948812 28468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:41.948590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.60:46713 (local address 127.0.0.1:56676)
0811 19:02:41.948710 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:41.948760 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.60:46713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260811 19:02:41.948940 28463 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.109.60:46713: Network error: Client connection negotiation failed: client connection to 127.13.109.60:46713: connect: Connection refused (error 111)
W20260811 19:02:41.952585 28471 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:02:41.952910 28471 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:02:41.953025 28471 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:02:41.957345 28471 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:02:41.957448 28471 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:02:41.957489 28471 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:02:41.957527 28471 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:02:41.957563 28471 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:02:41.963332 28471 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:38471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474792496492-13748-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474792496492-13748-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.13.109.62:40127,127.13.109.61:33979,127.13.109.60:46713
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.60.keytab
--principal=kudu/127.13.109.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.60:46713
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474792496492-13748-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.13.109.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28471
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474792496492-13748-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:02:41.964568 28471 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:02:41.965698 28471 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:02:41.974575 28476 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:02:41.974852 28471 server_base.cc:1034] running on GCE node
W20260811 19:02:41.974655 28479 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:02:41.974575 28477 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:02:41.975589 28471 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:02:41.976797 28471 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:02:41.977989 28471 hybrid_clock.cc:648] HybridClock initialized: now 1786474961977957 us; error 47 us; skew 500 ppm
Aug 11 19:02:41 dist-test-slave-0kls krb5kdc[28288](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474961, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:02:41.981108 28471 init.cc:377] Logged in from keytab as kudu/127.13.109.60@KRBTEST.COM (short username kudu)
I20260811 19:02:41.982206 28471 webserver.cc:460] Webserver started at http://127.13.109.60:42475/ using document root <none> and password file <none>
I20260811 19:02:41.982831 28471 fs_manager.cc:359] Metadata directory not provided
I20260811 19:02:41.982926 28471 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:02:41.983178 28471 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:02:41.985342 28471 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474792496492-13748-0/minicluster-data/master-2/data/instance:
uuid: "4e1c1d26c77748ac8de58201dbcf2231"
format_stamp: "Formatted at 2026-08-11 19:02:41 on dist-test-slave-0kls"
server_key: "6ab669cbab5e0ca18dfb824b981627b6"
server_key_iv: "33fc3837a7efda9d6e8471b26168f277"
server_key_version: "encryptionkey@0"
I20260811 19:02:41.986009 28471 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474792496492-13748-0/minicluster-data/master-2/wal/instance:
uuid: "4e1c1d26c77748ac8de58201dbcf2231"
format_stamp: "Formatted at 2026-08-11 19:02:41 on dist-test-slave-0kls"
server_key: "6ab669cbab5e0ca18dfb824b981627b6"
server_key_iv: "33fc3837a7efda9d6e8471b26168f277"
server_key_version: "encryptionkey@0"
I20260811 19:02:41.990358 28471 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 19:02:41.993064 28487 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:41.994464 28471 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 19:02:41.994624 28471 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474792496492-13748-0/minicluster-data/master-2/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474792496492-13748-0/minicluster-data/master-2/wal
uuid: "4e1c1d26c77748ac8de58201dbcf2231"
format_stamp: "Formatted at 2026-08-11 19:02:41 on dist-test-slave-0kls"
server_key: "6ab669cbab5e0ca18dfb824b981627b6"
server_key_iv: "33fc3837a7efda9d6e8471b26168f277"
server_key_version: "encryptionkey@0"
I20260811 19:02:41.994762 28471 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474792496492-13748-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474792496492-13748-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474792496492-13748-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:02:42.007444 28471 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:02:42.011811 28471 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:02:42.012038 28471 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:02:42.019073 28374 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.109.60" port: 46713 } attempt: 3
I20260811 19:02:42.019496 28471 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.60:46713
I20260811 19:02:42.019922 28543 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.60:46713 every 8 connection(s)
I20260811 19:02:42.020624 28471 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474792496492-13748-0/minicluster-data/master-2/data/info.pb
I20260811 19:02:42.024446 28545 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 19:02:42 dist-test-slave-0kls krb5kdc[28288](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474961, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.62@KRBTEST.COM for kudu/127.13.109.60@KRBTEST.COM
I20260811 19:02:42.028892 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 28471
I20260811 19:02:42.028999 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474792496492-13748-0/minicluster-data/master-2/wal/instance
I20260811 19:02:42.029302 13748 external_mini_cluster.cc:1428] Setting key 409c43e18174268ba7d1a861b23c0d9c
I20260811 19:02:42.029438 28545 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.109.62" port: 40127 } has no permanent_uuid. Determining permanent_uuid...
I20260811 19:02:42.034406 28379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:42.019659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.60:46713 (local address 127.0.0.1:56686)
0811 19:02:42.019843 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:42.019862 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 19:02:42.019968 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:02:42.023170 (+ 3202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:02:42.023174 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:02:42.023364 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:02:42.023479 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:42.023485 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:42.025304 (+ 1819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:42.025307 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:42.025943 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:42.025955 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:42.026940 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:42.026944 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:42.027020 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:42.027597 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:02:42.027612 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:02:42.029138 (+ 1526us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:02:42.032042 (+ 2904us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:02:42.032046 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:02:42.032048 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:02:42.032433 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:02:42.032859 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:02:42.032862 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:02:42.032864 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:02:42.032971 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:02:42.033774 (+ 803us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:02:42.033779 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:02:42.033940 (+ 161us) client_negotiation.cc:769] Sending connection context
0811 19:02:42.034051 (+ 111us) client_negotiation.cc:240] Negotiation successful
0811 19:02:42.034201 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":79}
I20260811 19:02:42.035564 28544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:42.020872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56686 (local address 127.13.109.60:46713)
0811 19:02:42.021321 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:42.021345 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:42.021408 (+ 63us) server_negotiation.cc:407] Connection header received
0811 19:02:42.022324 (+ 916us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:42.022348 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:42.022802 (+ 454us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:42.023211 (+ 409us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:02:42.023662 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:42.025160 (+ 1498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:42.026071 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:42.026769 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:42.026969 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:42.029413 (+ 2444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:02:42.029421 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:02:42.029441 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:02:42.029480 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:02:42.031902 (+ 2422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:42.032590 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:42.032596 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:42.032602 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:42.032749 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:42.033093 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:42.033096 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:42.033098 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:42.033646 (+ 548us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:02:42.033806 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:42.034223 (+ 417us) server_negotiation.cc:299] Negotiation successful
0811 19:02:42.034539 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":145,"threads_started":1}
Aug 11 19:02:42 dist-test-slave-0kls krb5kdc[28288](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474961, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.109.60@KRBTEST.COM
Aug 11 19:02:42 dist-test-slave-0kls krb5kdc[28288](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474961, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.60@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
I20260811 19:02:42.040118 28463 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.109.60" port: 46713 } attempt: 3
I20260811 19:02:42.040670 28374 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fce34686bc2a445c96c5226ba46b4c86: Bootstrap starting.
I20260811 19:02:42.042484 28377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:42.031661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35502 (local address 127.13.109.62:40127)
0811 19:02:42.031786 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:42.031790 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:42.032142 (+ 352us) server_negotiation.cc:407] Connection header received
0811 19:02:42.032478 (+ 336us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:42.032481 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:42.032542 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:42.032633 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:02:42.033462 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:42.034193 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:42.035395 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:42.035734 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:42.035815 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:42.038487 (+ 2672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:02:42.038491 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:02:42.038493 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:02:42.038522 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:02:42.040241 (+ 1719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:42.040852 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:42.040857 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:42.040859 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:42.040943 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:42.041283 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:42.041290 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:42.041291 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:42.041601 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:02:42.041688 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:42.042184 (+ 496us) server_negotiation.cc:299] Negotiation successful
0811 19:02:42.042316 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 19:02:42.043007 28548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:42.030459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56698 (local address 127.13.109.60:46713)
0811 19:02:42.030811 (+ 352us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:42.030820 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:42.030835 (+ 15us) server_negotiation.cc:407] Connection header received
0811 19:02:42.030969 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:42.030972 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:42.031018 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:42.031111 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:02:42.032034 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:42.032939 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:42.033915 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:42.034281 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:42.034376 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:42.037449 (+ 3073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:02:42.037453 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:02:42.037456 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:02:42.037485 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:02:42.039986 (+ 2501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:42.040885 (+ 899us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:42.040889 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:42.040890 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:42.040995 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:42.041442 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:42.041446 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:42.041447 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:42.041677 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:02:42.041891 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:42.042682 (+ 791us) server_negotiation.cc:299] Negotiation successful
0811 19:02:42.042805 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281,"thread_start_us":145,"threads_started":1}
I20260811 19:02:42.043143 28549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:42.031617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:40127 (local address 127.0.0.1:35502)
0811 19:02:42.032015 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:42.032067 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 19:02:42.032315 (+ 248us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:02:42.032674 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:02:42.032680 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:02:42.033178 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:02:42.033327 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:42.033334 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:42.034327 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:42.034330 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:42.035257 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:42.035268 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:42.035963 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:42.035966 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:42.036070 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:42.036584 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:02:42.036601 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:02:42.038362 (+ 1761us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:02:42.040386 (+ 2024us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:02:42.040400 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:02:42.040406 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:02:42.040732 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:02:42.041065 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:02:42.041071 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:02:42.041076 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:02:42.041172 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:02:42.041776 (+ 604us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:02:42.041812 (+ 36us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:02:42.041940 (+ 128us) client_negotiation.cc:769] Sending connection context
0811 19:02:42.042698 (+ 758us) client_negotiation.cc:240] Negotiation successful
0811 19:02:42.042910 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":102,"threads_started":1}
I20260811 19:02:42.044039 28374 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fce34686bc2a445c96c5226ba46b4c86: Neither blocks nor log segments found. Creating new log.
I20260811 19:02:42.044337 28545 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.109.61" port: 33979 } has no permanent_uuid. Determining permanent_uuid...
I20260811 19:02:42.045946 28374 log.cc:824] T 00000000000000000000000000000000 P fce34686bc2a445c96c5226ba46b4c86: Log is configured to *not* fsync() on all Append() calls
Aug 11 19:02:42 dist-test-slave-0kls krb5kdc[28288](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474961, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.61@KRBTEST.COM for kudu/127.13.109.60@KRBTEST.COM
I20260811 19:02:42.048604 28374 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fce34686bc2a445c96c5226ba46b4c86: No bootstrap required, opened a new log
Aug 11 19:02:42 dist-test-slave-0kls krb5kdc[28288](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474961, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.60@KRBTEST.COM for kudu/127.13.109.61@KRBTEST.COM
Aug 11 19:02:42 dist-test-slave-0kls krb5kdc[28288](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474961, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
I20260811 19:02:42.052718 28468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:42.040623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.60:46713 (local address 127.0.0.1:56710)
0811 19:02:42.040943 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:42.040960 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 19:02:42.041040 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:02:42.041609 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:02:42.041612 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:02:42.041846 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:02:42.041965 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:42.041973 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:42.043856 (+ 1883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:42.043860 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:42.044621 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:42.044635 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:42.045355 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:42.045359 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:42.045460 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:42.046796 (+ 1336us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:02:42.046812 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:02:42.048392 (+ 1580us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:02:42.050779 (+ 2387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:02:42.050785 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:02:42.050790 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:02:42.051108 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:02:42.051469 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:02:42.051473 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:02:42.051475 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:02:42.051576 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:02:42.052105 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:02:42.052108 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:02:42.052250 (+ 142us) client_negotiation.cc:769] Sending connection context
0811 19:02:42.052336 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 19:02:42.052497 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"mutex_wait_us":3}
I20260811 19:02:42.053478 28544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:42.041267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56710 (local address 127.13.109.60:46713)
0811 19:02:42.041386 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:42.041390 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:42.041402 (+ 12us) server_negotiation.cc:407] Connection header received
0811 19:02:42.041433 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:42.041435 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:42.041473 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:42.042087 (+ 614us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:02:42.042198 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:42.043720 (+ 1522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:42.044856 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:42.045216 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:42.045750 (+ 534us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:42.048534 (+ 2784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:02:42.048538 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:02:42.048541 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:02:42.048567 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:02:42.050620 (+ 2053us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:42.051257 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:42.051263 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:42.051268 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:42.051358 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:42.051719 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:42.051722 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:42.051723 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:42.051977 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:02:42.053055 (+ 1078us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:42.053166 (+ 111us) server_negotiation.cc:299] Negotiation successful
0811 19:02:42.053286 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 19:02:42.054050 28374 raft_consensus.cc:348] T 00000000000000000000000000000000 P fce34686bc2a445c96c5226ba46b4c86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fce34686bc2a445c96c5226ba46b4c86" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 40127 } } peers { permanent_uuid: "2399f3177b5f4f6aa611a35f49ca5301" member_type: VOTER last_known_addr { host: "127.13.109.61" port: 33979 } } peers { permanent_uuid: "4e1c1d26c77748ac8de58201dbcf2231" member_type: VOTER last_known_addr { host: "127.13.109.60" port: 46713 } }
I20260811 19:02:42.054360 28374 raft_consensus.cc:374] T 00000000000000000000000000000000 P fce34686bc2a445c96c5226ba46b4c86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:02:42.054426 28374 raft_consensus.cc:729] T 00000000000000000000000000000000 P fce34686bc2a445c96c5226ba46b4c86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fce34686bc2a445c96c5226ba46b4c86, State: Initialized, Role: FOLLOWER
I20260811 19:02:42.055035 28374 consensus_queue.cc:260] T 00000000000000000000000000000000 P fce34686bc2a445c96c5226ba46b4c86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fce34686bc2a445c96c5226ba46b4c86" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 40127 } } peers { permanent_uuid: "2399f3177b5f4f6aa611a35f49ca5301" member_type: VOTER last_known_addr { host: "127.13.109.61" port: 33979 } } peers { permanent_uuid: "4e1c1d26c77748ac8de58201dbcf2231" member_type: VOTER last_known_addr { host: "127.13.109.60" port: 46713 } }
I20260811 19:02:42.055740 28465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:42.044919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58404 (local address 127.13.109.61:33979)
0811 19:02:42.045051 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:42.045055 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:42.045179 (+ 124us) server_negotiation.cc:407] Connection header received
0811 19:02:42.045299 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:42.045304 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:42.045357 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:42.045449 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:02:42.045960 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:42.046625 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:42.047732 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:42.048022 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:42.048100 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:42.051377 (+ 3277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:02:42.051381 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:02:42.051384 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:02:42.051410 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:02:42.053653 (+ 2243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:42.054239 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:42.054243 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:42.054245 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:42.054335 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:42.054692 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:42.054695 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:42.054697 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:42.054965 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:02:42.055053 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:42.055436 (+ 383us) server_negotiation.cc:299] Negotiation successful
0811 19:02:42.055572 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":41,"server-negotiator.queue_time_us":54}
I20260811 19:02:42.056228 28549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:42.044819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.61:33979 (local address 127.0.0.1:58404)
0811 19:02:42.045112 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:42.045125 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 19:02:42.045216 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:02:42.045459 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:02:42.045461 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:02:42.045689 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:02:42.045854 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:42.045864 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:42.046769 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:42.046772 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:42.047564 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:42.047576 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:42.048273 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:42.048276 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:42.048380 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:42.049088 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:02:42.049104 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:02:42.051204 (+ 2100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:02:42.053822 (+ 2618us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:02:42.053825 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:02:42.053827 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:02:42.054118 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:02:42.054463 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:02:42.054467 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:02:42.054472 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:02:42.054564 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:02:42.055122 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:02:42.055124 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:02:42.055227 (+ 103us) client_negotiation.cc:769] Sending connection context
0811 19:02:42.055925 (+ 698us) client_negotiation.cc:240] Negotiation successful
0811 19:02:42.056074 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":109,"threads_started":1}
I20260811 19:02:42.056958 28374 sys_catalog.cc:564] T 00000000000000000000000000000000 P fce34686bc2a445c96c5226ba46b4c86 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:02:42.056944 28545 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.109.60" port: 46713 } has no permanent_uuid. Determining permanent_uuid...
I20260811 19:02:42.057183 28463 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2399f3177b5f4f6aa611a35f49ca5301: Bootstrap starting.
I20260811 19:02:42.057315 28377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:42.044425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35516 (local address 127.13.109.62:40127)
0811 19:02:42.044533 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:42.044538 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:42.044747 (+ 209us) server_negotiation.cc:407] Connection header received
0811 19:02:42.044944 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:42.044948 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:42.045001 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:42.045054 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:02:42.046439 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:42.047134 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:42.048452 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:42.048820 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:42.048901 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:42.052295 (+ 3394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:02:42.052299 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:02:42.052302 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:02:42.052332 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:02:42.054341 (+ 2009us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:42.055022 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:42.055025 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:42.055027 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:42.055164 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:42.055600 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:42.055604 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:42.055605 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:42.055846 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:02:42.056015 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:42.057045 (+ 1030us) server_negotiation.cc:299] Negotiation successful
0811 19:02:42.057174 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 19:02:42.060578 28463 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2399f3177b5f4f6aa611a35f49ca5301: Neither blocks nor log segments found. Creating new log.
I20260811 19:02:42.061595 28463 log.cc:824] T 00000000000000000000000000000000 P 2399f3177b5f4f6aa611a35f49ca5301: Log is configured to *not* fsync() on all Append() calls
Aug 11 19:02:42 dist-test-slave-0kls krb5kdc[28288](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474961, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.60@KRBTEST.COM for kudu/127.13.109.60@KRBTEST.COM
I20260811 19:02:42.064018 28463 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2399f3177b5f4f6aa611a35f49ca5301: No bootstrap required, opened a new log
Aug 11 19:02:42 dist-test-slave-0kls krb5kdc[28288](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474961, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.109.61@KRBTEST.COM
I20260811 19:02:42.064033 28553 sys_catalog.cc:455] T 00000000000000000000000000000000 P fce34686bc2a445c96c5226ba46b4c86 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fce34686bc2a445c96c5226ba46b4c86" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 40127 } } peers { permanent_uuid: "2399f3177b5f4f6aa611a35f49ca5301" member_type: VOTER last_known_addr { host: "127.13.109.61" port: 33979 } } peers { permanent_uuid: "4e1c1d26c77748ac8de58201dbcf2231" member_type: VOTER last_known_addr { host: "127.13.109.60" port: 46713 } } }
I20260811 19:02:42.067384 28553 sys_catalog.cc:458] T 00000000000000000000000000000000 P fce34686bc2a445c96c5226ba46b4c86 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 19:02:42.062973 28301 master_runner.cc:186] Error getting master registration for 127.13.109.61:33979: OK, instance_id { permanent_uuid: "2399f3177b5f4f6aa611a35f49ca5301" instance_seqno: 1786474961787165 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 19:02:42.067874 28301 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 19:02:42.068249 28549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:42.057443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.60:46713 (local address 127.0.0.1:56712)
0811 19:02:42.057831 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:42.057848 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 19:02:42.057934 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:02:42.058375 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:02:42.058379 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:02:42.058558 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:02:42.058663 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:42.058671 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:42.060012 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:42.060015 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:42.060774 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:42.060786 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:42.061460 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:42.061463 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:42.061536 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:42.062121 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:02:42.062135 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:02:42.063731 (+ 1596us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:02:42.066159 (+ 2428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:02:42.066162 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:02:42.066164 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:02:42.066395 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:02:42.066673 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:02:42.066676 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:02:42.066678 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:02:42.067021 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:02:42.067752 (+ 731us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:02:42.067754 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:02:42.067849 (+ 95us) client_negotiation.cc:769] Sending connection context
0811 19:02:42.067956 (+ 107us) client_negotiation.cc:240] Negotiation successful
0811 19:02:42.068108 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276}
I20260811 19:02:42.068657 28553 raft_consensus.cc:482] T 00000000000000000000000000000000 P fce34686bc2a445c96c5226ba46b4c86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:02:42.068907 28548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:42.057655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56712 (local address 127.13.109.60:46713)
0811 19:02:42.058088 (+ 433us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:42.058093 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:42.058107 (+ 14us) server_negotiation.cc:407] Connection header received
0811 19:02:42.058141 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:42.058146 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:42.058191 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:42.058265 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:02:42.058897 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:42.059783 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:42.060895 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:42.061244 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:42.061344 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:42.064156 (+ 2812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:02:42.064160 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:02:42.064163 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:02:42.064188 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:02:42.066044 (+ 1856us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:42.066487 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:42.066491 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:42.066493 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:42.066572 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:42.067125 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:42.067128 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:42.067129 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:42.067617 (+ 488us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:02:42.067715 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:42.068594 (+ 879us) server_negotiation.cc:299] Negotiation successful
0811 19:02:42.068741 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360}
I20260811 19:02:42.068861 28463 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2399f3177b5f4f6aa611a35f49ca5301 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fce34686bc2a445c96c5226ba46b4c86" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 40127 } } peers { permanent_uuid: "2399f3177b5f4f6aa611a35f49ca5301" member_type: VOTER last_known_addr { host: "127.13.109.61" port: 33979 } } peers { permanent_uuid: "4e1c1d26c77748ac8de58201dbcf2231" member_type: VOTER last_known_addr { host: "127.13.109.60" port: 46713 } }
I20260811 19:02:42.069113 28463 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2399f3177b5f4f6aa611a35f49ca5301 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:02:42.069370 28463 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2399f3177b5f4f6aa611a35f49ca5301 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2399f3177b5f4f6aa611a35f49ca5301, State: Initialized, Role: FOLLOWER
I20260811 19:02:42.069497 28553 raft_consensus.cc:504] T 00000000000000000000000000000000 P fce34686bc2a445c96c5226ba46b4c86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fce34686bc2a445c96c5226ba46b4c86" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 40127 } } peers { permanent_uuid: "2399f3177b5f4f6aa611a35f49ca5301" member_type: VOTER last_known_addr { host: "127.13.109.61" port: 33979 } } peers { permanent_uuid: "4e1c1d26c77748ac8de58201dbcf2231" member_type: VOTER last_known_addr { host: "127.13.109.60" port: 46713 } }
I20260811 19:02:42.070024 28463 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2399f3177b5f4f6aa611a35f49ca5301 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fce34686bc2a445c96c5226ba46b4c86" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 40127 } } peers { permanent_uuid: "2399f3177b5f4f6aa611a35f49ca5301" member_type: VOTER last_known_addr { host: "127.13.109.61" port: 33979 } } peers { permanent_uuid: "4e1c1d26c77748ac8de58201dbcf2231" member_type: VOTER last_known_addr { host: "127.13.109.60" port: 46713 } }
I20260811 19:02:42.070345 28465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:42.059333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58406 (local address 127.13.109.61:33979)
0811 19:02:42.059508 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:42.059512 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:42.059721 (+ 209us) server_negotiation.cc:407] Connection header received
0811 19:02:42.059921 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:42.059924 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:42.059968 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:42.060016 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:02:42.061078 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:42.061864 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:42.062945 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:42.063254 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:42.063331 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:42.066458 (+ 3127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:02:42.066464 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:02:42.066468 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:02:42.066497 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:02:42.068231 (+ 1734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:42.068748 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:42.068752 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:42.068754 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:42.068849 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:42.069196 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:42.069199 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:42.069201 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:42.069456 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:02:42.069633 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:42.070052 (+ 419us) server_negotiation.cc:299] Negotiation successful
0811 19:02:42.070199 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
I20260811 19:02:42.071583 28463 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2399f3177b5f4f6aa611a35f49ca5301 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:02:42.071802 28435 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fce34686bc2a445c96c5226ba46b4c86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2399f3177b5f4f6aa611a35f49ca5301" is_pre_election: true
I20260811 19:02:42.072065 28435 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2399f3177b5f4f6aa611a35f49ca5301 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fce34686bc2a445c96c5226ba46b4c86 in term 0.
I20260811 19:02:42.072302 28545 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e1c1d26c77748ac8de58201dbcf2231: Bootstrap starting.
I20260811 19:02:42.072557 28553 leader_election.cc:288] T 00000000000000000000000000000000 P fce34686bc2a445c96c5226ba46b4c86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2399f3177b5f4f6aa611a35f49ca5301 (127.13.109.61:33979), 4e1c1d26c77748ac8de58201dbcf2231 (127.13.109.60:46713)
I20260811 19:02:42.072638 28318 leader_election.cc:302] T 00000000000000000000000000000000 P fce34686bc2a445c96c5226ba46b4c86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2399f3177b5f4f6aa611a35f49ca5301, fce34686bc2a445c96c5226ba46b4c86; no voters:
I20260811 19:02:42.072934 28553 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fce34686bc2a445c96c5226ba46b4c86 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 19:02:42.073099 28553 raft_consensus.cc:482] T 00000000000000000000000000000000 P fce34686bc2a445c96c5226ba46b4c86 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 19:02:42.073200 28553 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fce34686bc2a445c96c5226ba46b4c86 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:02:42.074610 28553 raft_consensus.cc:504] T 00000000000000000000000000000000 P fce34686bc2a445c96c5226ba46b4c86 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fce34686bc2a445c96c5226ba46b4c86" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 40127 } } peers { permanent_uuid: "2399f3177b5f4f6aa611a35f49ca5301" member_type: VOTER last_known_addr { host: "127.13.109.61" port: 33979 } } peers { permanent_uuid: "4e1c1d26c77748ac8de58201dbcf2231" member_type: VOTER last_known_addr { host: "127.13.109.60" port: 46713 } }
I20260811 19:02:42.074980 13748 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 19:02:42.075441 28435 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fce34686bc2a445c96c5226ba46b4c86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2399f3177b5f4f6aa611a35f49ca5301"
I20260811 19:02:42.075595 28435 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2399f3177b5f4f6aa611a35f49ca5301 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:02:42.076972 28435 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2399f3177b5f4f6aa611a35f49ca5301 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fce34686bc2a445c96c5226ba46b4c86 in term 1.
I20260811 19:02:42.077325 28318 leader_election.cc:302] T 00000000000000000000000000000000 P fce34686bc2a445c96c5226ba46b4c86 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2399f3177b5f4f6aa611a35f49ca5301, fce34686bc2a445c96c5226ba46b4c86; no voters:
I20260811 19:02:42.077549 28567 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fce34686bc2a445c96c5226ba46b4c86 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:02:42.077836 28567 raft_consensus.cc:686] T 00000000000000000000000000000000 P fce34686bc2a445c96c5226ba46b4c86 [term 1 LEADER]: Becoming Leader. State: Replica: fce34686bc2a445c96c5226ba46b4c86, State: Running, Role: LEADER
I20260811 19:02:42.078233 28567 consensus_queue.cc:237] T 00000000000000000000000000000000 P fce34686bc2a445c96c5226ba46b4c86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fce34686bc2a445c96c5226ba46b4c86" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 40127 } } peers { permanent_uuid: "2399f3177b5f4f6aa611a35f49ca5301" member_type: VOTER last_known_addr { host: "127.13.109.61" port: 33979 } } peers { permanent_uuid: "4e1c1d26c77748ac8de58201dbcf2231" member_type: VOTER last_known_addr { host: "127.13.109.60" port: 46713 } }
I20260811 19:02:42.079710 28545 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4e1c1d26c77748ac8de58201dbcf2231: Neither blocks nor log segments found. Creating new log.
I20260811 19:02:42.080081 28553 leader_election.cc:288] T 00000000000000000000000000000000 P fce34686bc2a445c96c5226ba46b4c86 [CANDIDATE]: Term 1 election: Requested vote from peers 2399f3177b5f4f6aa611a35f49ca5301 (127.13.109.61:33979), 4e1c1d26c77748ac8de58201dbcf2231 (127.13.109.60:46713)
I20260811 19:02:42.080691 28545 log.cc:824] T 00000000000000000000000000000000 P 4e1c1d26c77748ac8de58201dbcf2231: Log is configured to *not* fsync() on all Append() calls
I20260811 19:02:42.080987 28581 sys_catalog.cc:455] T 00000000000000000000000000000000 P fce34686bc2a445c96c5226ba46b4c86 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fce34686bc2a445c96c5226ba46b4c86. Latest consensus state: current_term: 1 leader_uuid: "fce34686bc2a445c96c5226ba46b4c86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fce34686bc2a445c96c5226ba46b4c86" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 40127 } } peers { permanent_uuid: "2399f3177b5f4f6aa611a35f49ca5301" member_type: VOTER last_known_addr { host: "127.13.109.61" port: 33979 } } peers { permanent_uuid: "4e1c1d26c77748ac8de58201dbcf2231" member_type: VOTER last_known_addr { host: "127.13.109.60" port: 46713 } } }
I20260811 19:02:42.081125 28581 sys_catalog.cc:458] T 00000000000000000000000000000000 P fce34686bc2a445c96c5226ba46b4c86 [sys.catalog]: This master's current role is: LEADER
I20260811 19:02:42.082029 28568 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2399f3177b5f4f6aa611a35f49ca5301 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fce34686bc2a445c96c5226ba46b4c86" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 40127 } } peers { permanent_uuid: "2399f3177b5f4f6aa611a35f49ca5301" member_type: VOTER last_known_addr { host: "127.13.109.61" port: 33979 } } peers { permanent_uuid: "4e1c1d26c77748ac8de58201dbcf2231" member_type: VOTER last_known_addr { host: "127.13.109.60" port: 46713 } } }
I20260811 19:02:42.082309 28568 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2399f3177b5f4f6aa611a35f49ca5301 [sys.catalog]: This master's current role is: FOLLOWER
W20260811 19:02:42.083799 28565 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fce34686bc2a445c96c5226ba46b4c86: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 19:02:42.083874 28565 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 19:02:42.083988 28593 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:02:42.085498 28593 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:02:42.087044 28385 master_runner.cc:186] Error getting master registration for 127.13.109.60:46713: OK, instance_id { permanent_uuid: "4e1c1d26c77748ac8de58201dbcf2231" instance_seqno: 1786474962017239 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 19:02:42.091421 28385 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260811 19:02:42.092073 28591 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2399f3177b5f4f6aa611a35f49ca5301: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 19:02:42.092180 28591 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 19:02:42.092958 28545 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e1c1d26c77748ac8de58201dbcf2231: No bootstrap required, opened a new log
I20260811 19:02:42.094058 28435 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2399f3177b5f4f6aa611a35f49ca5301 [term 1 FOLLOWER]: Refusing update from remote peer fce34686bc2a445c96c5226ba46b4c86: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:02:42.094871 28581 consensus_queue.cc:1035] T 00000000000000000000000000000000 P fce34686bc2a445c96c5226ba46b4c86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2399f3177b5f4f6aa611a35f49ca5301" member_type: VOTER last_known_addr { host: "127.13.109.61" port: 33979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:02:42.097419 28545 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4e1c1d26c77748ac8de58201dbcf2231 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fce34686bc2a445c96c5226ba46b4c86" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 40127 } } peers { permanent_uuid: "2399f3177b5f4f6aa611a35f49ca5301" member_type: VOTER last_known_addr { host: "127.13.109.61" port: 33979 } } peers { permanent_uuid: "4e1c1d26c77748ac8de58201dbcf2231" member_type: VOTER last_known_addr { host: "127.13.109.60" port: 46713 } }
I20260811 19:02:42.097765 28545 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4e1c1d26c77748ac8de58201dbcf2231 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:02:42.097918 28545 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4e1c1d26c77748ac8de58201dbcf2231 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e1c1d26c77748ac8de58201dbcf2231, State: Initialized, Role: FOLLOWER
I20260811 19:02:42.098534 28545 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4e1c1d26c77748ac8de58201dbcf2231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fce34686bc2a445c96c5226ba46b4c86" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 40127 } } peers { permanent_uuid: "2399f3177b5f4f6aa611a35f49ca5301" member_type: VOTER last_known_addr { host: "127.13.109.61" port: 33979 } } peers { permanent_uuid: "4e1c1d26c77748ac8de58201dbcf2231" member_type: VOTER last_known_addr { host: "127.13.109.60" port: 46713 } }
I20260811 19:02:42.100288 28545 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4e1c1d26c77748ac8de58201dbcf2231 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:02:42.100363 28514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fce34686bc2a445c96c5226ba46b4c86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e1c1d26c77748ac8de58201dbcf2231"
I20260811 19:02:42.100917 28515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fce34686bc2a445c96c5226ba46b4c86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e1c1d26c77748ac8de58201dbcf2231" is_pre_election: true
I20260811 19:02:42.100996 28513 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4e1c1d26c77748ac8de58201dbcf2231 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:02:42.102447 28568 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2399f3177b5f4f6aa611a35f49ca5301 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fce34686bc2a445c96c5226ba46b4c86. Latest consensus state: current_term: 1 leader_uuid: "fce34686bc2a445c96c5226ba46b4c86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fce34686bc2a445c96c5226ba46b4c86" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 40127 } } peers { permanent_uuid: "2399f3177b5f4f6aa611a35f49ca5301" member_type: VOTER last_known_addr { host: "127.13.109.61" port: 33979 } } peers { permanent_uuid: "4e1c1d26c77748ac8de58201dbcf2231" member_type: VOTER last_known_addr { host: "127.13.109.60" port: 46713 } } }
I20260811 19:02:42.102638 28568 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2399f3177b5f4f6aa611a35f49ca5301 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 19:02:42.103210 28513 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4e1c1d26c77748ac8de58201dbcf2231 [term 1 FOLLOWER]: Refusing update from remote peer fce34686bc2a445c96c5226ba46b4c86: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:02:42.103904 28581 consensus_queue.cc:1035] T 00000000000000000000000000000000 P fce34686bc2a445c96c5226ba46b4c86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e1c1d26c77748ac8de58201dbcf2231" member_type: VOTER last_known_addr { host: "127.13.109.60" port: 46713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:02:42.108855 28465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:42.077379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58422 (local address 127.13.109.61:33979)
0811 19:02:42.077510 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:42.077517 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:42.090470 (+ 12953us) server_negotiation.cc:407] Connection header received
0811 19:02:42.092344 (+ 1874us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:42.092348 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:42.092400 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:42.092456 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:02:42.093516 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:42.094279 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:42.096940 (+ 2661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:42.097253 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:42.097342 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:42.104710 (+ 7368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:02:42.104715 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:02:42.104718 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:02:42.104757 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:02:42.106677 (+ 1920us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:42.107266 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:42.107271 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:42.107272 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:42.107371 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:42.107716 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:42.107720 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:42.107722 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:42.107990 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:02:42.108112 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:42.108505 (+ 393us) server_negotiation.cc:299] Negotiation successful
0811 19:02:42.108652 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 19:02:42.110060 28598 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e1c1d26c77748ac8de58201dbcf2231 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fce34686bc2a445c96c5226ba46b4c86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fce34686bc2a445c96c5226ba46b4c86" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 40127 } } peers { permanent_uuid: "2399f3177b5f4f6aa611a35f49ca5301" member_type: VOTER last_known_addr { host: "127.13.109.61" port: 33979 } } peers { permanent_uuid: "4e1c1d26c77748ac8de58201dbcf2231" member_type: VOTER last_known_addr { host: "127.13.109.60" port: 46713 } } }
I20260811 19:02:42.110248 28598 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e1c1d26c77748ac8de58201dbcf2231 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 19:02:42.110378 28598 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e1c1d26c77748ac8de58201dbcf2231 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fce34686bc2a445c96c5226ba46b4c86. Latest consensus state: current_term: 1 leader_uuid: "fce34686bc2a445c96c5226ba46b4c86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fce34686bc2a445c96c5226ba46b4c86" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 40127 } } peers { permanent_uuid: "2399f3177b5f4f6aa611a35f49ca5301" member_type: VOTER last_known_addr { host: "127.13.109.61" port: 33979 } } peers { permanent_uuid: "4e1c1d26c77748ac8de58201dbcf2231" member_type: VOTER last_known_addr { host: "127.13.109.60" port: 46713 } } }
I20260811 19:02:42.110472 28598 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e1c1d26c77748ac8de58201dbcf2231 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 19:02:42.115726 28548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:42.077891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56714 (local address 127.13.109.60:46713)
0811 19:02:42.081212 (+ 3321us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:42.081219 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:42.096992 (+ 15773us) server_negotiation.cc:407] Connection header received
0811 19:02:42.097211 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:42.097214 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:42.097269 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:42.097322 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:02:42.098527 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:42.099353 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:42.100478 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:42.100829 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:42.100911 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:42.111049 (+ 10138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:02:42.111054 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:02:42.111057 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:02:42.111087 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:02:42.113513 (+ 2426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:42.114124 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:42.114129 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:42.114130 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:42.114230 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:42.114567 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:42.114571 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:42.114573 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:42.114824 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:02:42.114949 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:42.115335 (+ 386us) server_negotiation.cc:299] Negotiation successful
0811 19:02:42.115497 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3231}
I20260811 19:02:42.120313 28377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:42.090763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35520 (local address 127.13.109.62:40127)
0811 19:02:42.090875 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:42.090880 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:42.110229 (+ 19349us) server_negotiation.cc:407] Connection header received
0811 19:02:42.110856 (+ 627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:42.110860 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:42.110920 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:42.110975 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:02:42.112759 (+ 1784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:42.113342 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:42.114292 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:42.114569 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:42.114687 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:42.116714 (+ 2027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:02:42.116718 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:02:42.116720 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:02:42.116754 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:02:42.118454 (+ 1700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:42.118945 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:42.118949 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:42.118951 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:42.119046 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:42.119337 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:42.119340 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:42.119342 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:42.119563 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:02:42.119747 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:42.120001 (+ 254us) server_negotiation.cc:299] Negotiation successful
0811 19:02:42.120135 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
W20260811 19:02:42.123989 28612 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4e1c1d26c77748ac8de58201dbcf2231: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 19:02:42.124109 28612 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 19:02:42.127429 28553 sys_catalog.cc:455] T 00000000000000000000000000000000 P fce34686bc2a445c96c5226ba46b4c86 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "fce34686bc2a445c96c5226ba46b4c86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fce34686bc2a445c96c5226ba46b4c86" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 40127 } } peers { permanent_uuid: "2399f3177b5f4f6aa611a35f49ca5301" member_type: VOTER last_known_addr { host: "127.13.109.61" port: 33979 } } peers { permanent_uuid: "4e1c1d26c77748ac8de58201dbcf2231" member_type: VOTER last_known_addr { host: "127.13.109.60" port: 46713 } } }
I20260811 19:02:42.127583 28553 sys_catalog.cc:458] T 00000000000000000000000000000000 P fce34686bc2a445c96c5226ba46b4c86 [sys.catalog]: This master's current role is: LEADER
I20260811 19:02:42.127715 28594 mvcc.cc:205] Tried to move back new op lower bound from 7317401444726173696 to 7317401444675727360. Current Snapshot: MvccSnapshot[applied={T|T < 7317401444726173696}]
I20260811 19:02:42.128683 28568 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2399f3177b5f4f6aa611a35f49ca5301 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "fce34686bc2a445c96c5226ba46b4c86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fce34686bc2a445c96c5226ba46b4c86" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 40127 } } peers { permanent_uuid: "2399f3177b5f4f6aa611a35f49ca5301" member_type: VOTER last_known_addr { host: "127.13.109.61" port: 33979 } } peers { permanent_uuid: "4e1c1d26c77748ac8de58201dbcf2231" member_type: VOTER last_known_addr { host: "127.13.109.60" port: 46713 } } }
I20260811 19:02:42.128835 28568 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2399f3177b5f4f6aa611a35f49ca5301 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 19:02:42.129515 28581 sys_catalog.cc:455] T 00000000000000000000000000000000 P fce34686bc2a445c96c5226ba46b4c86 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "fce34686bc2a445c96c5226ba46b4c86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fce34686bc2a445c96c5226ba46b4c86" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 40127 } } peers { permanent_uuid: "2399f3177b5f4f6aa611a35f49ca5301" member_type: VOTER last_known_addr { host: "127.13.109.61" port: 33979 } } peers { permanent_uuid: "4e1c1d26c77748ac8de58201dbcf2231" member_type: VOTER last_known_addr { host: "127.13.109.60" port: 46713 } } }
I20260811 19:02:42.129642 28581 sys_catalog.cc:458] T 00000000000000000000000000000000 P fce34686bc2a445c96c5226ba46b4c86 [sys.catalog]: This master's current role is: LEADER
I20260811 19:02:42.130637 28614 mvcc.cc:205] Tried to move back new op lower bound from 7317401444726173696 to 7317401444675727360. Current Snapshot: MvccSnapshot[applied={T|T < 7317401444726173696}]
I20260811 19:02:42.130831 28613 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e1c1d26c77748ac8de58201dbcf2231 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "fce34686bc2a445c96c5226ba46b4c86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fce34686bc2a445c96c5226ba46b4c86" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 40127 } } peers { permanent_uuid: "2399f3177b5f4f6aa611a35f49ca5301" member_type: VOTER last_known_addr { host: "127.13.109.61" port: 33979 } } peers { permanent_uuid: "4e1c1d26c77748ac8de58201dbcf2231" member_type: VOTER last_known_addr { host: "127.13.109.60" port: 46713 } } }
I20260811 19:02:42.130967 28613 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e1c1d26c77748ac8de58201dbcf2231 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 19:02:42.139899 28593 catalog_manager.cc:1347] Generated new cluster ID: 79c93a099a3e4124ad2b67d504f169db
I20260811 19:02:42.140008 28593 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:02:42.156648 28593 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:02:42.158046 28593 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:02:42.176774 28593 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fce34686bc2a445c96c5226ba46b4c86: Generated new TSK 0
I20260811 19:02:42.177871 28593 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 19:02:43.093647 28591 catalog_manager.cc:1259] Loaded cluster ID: 79c93a099a3e4124ad2b67d504f169db
I20260811 19:02:43.093773 28591 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2399f3177b5f4f6aa611a35f49ca5301: loading cluster ID for follower catalog manager: success
I20260811 19:02:43.095506 28591 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2399f3177b5f4f6aa611a35f49ca5301: acquiring CA information for follower catalog manager: success
I20260811 19:02:43.096798 28591 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2399f3177b5f4f6aa611a35f49ca5301: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 19:02:43.125604 28612 catalog_manager.cc:1259] Loaded cluster ID: 79c93a099a3e4124ad2b67d504f169db
I20260811 19:02:43.125763 28612 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 4e1c1d26c77748ac8de58201dbcf2231: loading cluster ID for follower catalog manager: success
I20260811 19:02:43.127718 28612 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 4e1c1d26c77748ac8de58201dbcf2231: acquiring CA information for follower catalog manager: success
I20260811 19:02:43.129133 28612 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 4e1c1d26c77748ac8de58201dbcf2231: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 19:02:43.252246 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 28301
I20260811 19:02:43.262282 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 28385
I20260811 19:02:43.271209 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 28471
2026-08-11T19:02:43Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3153 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474792496492-13748-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:02:43 dist-test-slave-0kls krb5kdc[28627](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:02:43 dist-test-slave-0kls krb5kdc[28627](info): set up 2 sockets
Aug 11 19:02:43 dist-test-slave-0kls krb5kdc[28627](info): commencing operation
krb5kdc: starting...
W20260811 19:02:44.383006 13748 mini_kdc.cc:121] Time spent starting KDC: real 1.086s user 0.001s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474792496492-13748-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474792496492-13748-0/krb5kdc/test-user.keytab.
Aug 11 19:02:44 dist-test-slave-0kls krb5kdc[28627](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474964, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:02:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:02:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.109.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.109.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.109.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab.
Entry for principal kudu/127.13.109.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab.
I20260811 19:02:44.526503 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474792496492-13748-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474792496492-13748-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474792496492-13748-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.62:33739
--webserver_interface=127.13.109.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab
--principal=kudu/127.13.109.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:45379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.109.62:33739
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:02:44.647384 28640 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:02:44.647771 28640 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:02:44.647862 28640 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:02:44.652109 28640 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:02:44.652236 28640 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:02:44.652274 28640 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:02:44.652318 28640 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:02:44.652355 28640 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:02:44.658068 28640 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:45379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474792496492-13748-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474792496492-13748-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.109.62:33739
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab
--principal=kudu/127.13.109.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.62:33739
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.109.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28640
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474792496492-13748-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:02:44.659368 28640 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:02:44.660524 28640 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:02:44.670219 28649 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:02:44.670389 28640 server_base.cc:1034] running on GCE node
W20260811 19:02:44.670234 28646 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:02:44.670240 28647 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:02:44.671200 28640 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:02:44.672416 28640 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:02:44.673619 28640 hybrid_clock.cc:648] HybridClock initialized: now 1786474964673588 us; error 52 us; skew 500 ppm
Aug 11 19:02:44 dist-test-slave-0kls krb5kdc[28627](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474964, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:02:44.676692 28640 init.cc:377] Logged in from keytab as kudu/127.13.109.62@KRBTEST.COM (short username kudu)
I20260811 19:02:44.677727 28640 webserver.cc:460] Webserver started at http://127.13.109.62:43207/ using document root <none> and password file <none>
I20260811 19:02:44.678391 28640 fs_manager.cc:359] Metadata directory not provided
I20260811 19:02:44.678489 28640 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:02:44.678752 28640 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:02:44.681053 28640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474792496492-13748-0/minicluster-data/master-0/data/instance:
uuid: "13bd21dfb6c24c24b9ca3519a7d860a5"
format_stamp: "Formatted at 2026-08-11 19:02:44 on dist-test-slave-0kls"
server_key: "b2b42b0c7b6311eae1fe2668235035fa"
server_key_iv: "30aedeac2cf8939d6e6a84cae5d15ff4"
server_key_version: "encryptionkey@0"
I20260811 19:02:44.682101 28640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474792496492-13748-0/minicluster-data/master-0/wal/instance:
uuid: "13bd21dfb6c24c24b9ca3519a7d860a5"
format_stamp: "Formatted at 2026-08-11 19:02:44 on dist-test-slave-0kls"
server_key: "b2b42b0c7b6311eae1fe2668235035fa"
server_key_iv: "30aedeac2cf8939d6e6a84cae5d15ff4"
server_key_version: "encryptionkey@0"
I20260811 19:02:44.686712 28640 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 19:02:44.689610 28656 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:44.691135 28640 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 19:02:44.691332 28640 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474792496492-13748-0/minicluster-data/master-0/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474792496492-13748-0/minicluster-data/master-0/wal
uuid: "13bd21dfb6c24c24b9ca3519a7d860a5"
format_stamp: "Formatted at 2026-08-11 19:02:44 on dist-test-slave-0kls"
server_key: "b2b42b0c7b6311eae1fe2668235035fa"
server_key_iv: "30aedeac2cf8939d6e6a84cae5d15ff4"
server_key_version: "encryptionkey@0"
I20260811 19:02:44.691493 28640 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474792496492-13748-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474792496492-13748-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474792496492-13748-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:02:44.712236 28640 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:02:44.715792 28640 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:02:44.716032 28640 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:02:44.723938 28640 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.62:33739
I20260811 19:02:44.723968 28716 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.62:33739 every 8 connection(s)
I20260811 19:02:44.724984 28640 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
I20260811 19:02:44.725765 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 28640
I20260811 19:02:44.725915 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474792496492-13748-0/minicluster-data/master-0/wal/instance
I20260811 19:02:44.726256 13748 external_mini_cluster.cc:1428] Setting key 989e012651493bc0cbd40c42097a1fd0
I20260811 19:02:44.728541 28717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:02:44.735119 28717 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13bd21dfb6c24c24b9ca3519a7d860a5: Bootstrap starting.
Aug 11 19:02:44 dist-test-slave-0kls krb5kdc[28627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474964, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
I20260811 19:02:44.737758 28717 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 13bd21dfb6c24c24b9ca3519a7d860a5: Neither blocks nor log segments found. Creating new log.
I20260811 19:02:44.738772 28717 log.cc:824] T 00000000000000000000000000000000 P 13bd21dfb6c24c24b9ca3519a7d860a5: Log is configured to *not* fsync() on all Append() calls
I20260811 19:02:44.741369 28717 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13bd21dfb6c24c24b9ca3519a7d860a5: No bootstrap required, opened a new log
I20260811 19:02:44.742525 28721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:44.727752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37004 (local address 127.13.109.62:33739)
0811 19:02:44.728362 (+ 610us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:44.728380 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:44.728421 (+ 41us) server_negotiation.cc:407] Connection header received
0811 19:02:44.729205 (+ 784us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:44.729224 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:44.729582 (+ 358us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:44.730009 (+ 427us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:02:44.730915 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:44.732031 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:44.733340 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:44.734127 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:44.734290 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:44.737022 (+ 2732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:02:44.737029 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:02:44.737046 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:02:44.737083 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:02:44.739207 (+ 2124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:44.739818 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:44.739824 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:44.739830 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:44.739966 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:44.740245 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:44.740248 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:44.740250 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:44.740776 (+ 526us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:02:44.740964 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:44.741210 (+ 246us) server_negotiation.cc:299] Negotiation successful
0811 19:02:44.741519 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":447,"thread_start_us":153,"threads_started":1}
I20260811 19:02:44.745591 13748 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 19:02:44.745939 28717 raft_consensus.cc:348] T 00000000000000000000000000000000 P 13bd21dfb6c24c24b9ca3519a7d860a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13bd21dfb6c24c24b9ca3519a7d860a5" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 33739 } }
I20260811 19:02:44.746178 28717 raft_consensus.cc:374] T 00000000000000000000000000000000 P 13bd21dfb6c24c24b9ca3519a7d860a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:02:44.746258 28717 raft_consensus.cc:729] T 00000000000000000000000000000000 P 13bd21dfb6c24c24b9ca3519a7d860a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13bd21dfb6c24c24b9ca3519a7d860a5, State: Initialized, Role: FOLLOWER
I20260811 19:02:44.746783 28717 consensus_queue.cc:260] T 00000000000000000000000000000000 P 13bd21dfb6c24c24b9ca3519a7d860a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13bd21dfb6c24c24b9ca3519a7d860a5" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 33739 } }
I20260811 19:02:44.746937 28717 raft_consensus.cc:388] T 00000000000000000000000000000000 P 13bd21dfb6c24c24b9ca3519a7d860a5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:02:44.747016 28717 raft_consensus.cc:482] T 00000000000000000000000000000000 P 13bd21dfb6c24c24b9ca3519a7d860a5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:02:44.747118 28717 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 13bd21dfb6c24c24b9ca3519a7d860a5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:02:44.748209 28717 raft_consensus.cc:504] T 00000000000000000000000000000000 P 13bd21dfb6c24c24b9ca3519a7d860a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13bd21dfb6c24c24b9ca3519a7d860a5" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 33739 } }
I20260811 19:02:44.748586 28717 leader_election.cc:302] T 00000000000000000000000000000000 P 13bd21dfb6c24c24b9ca3519a7d860a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 13bd21dfb6c24c24b9ca3519a7d860a5; no voters:
I20260811 19:02:44.748966 28717 leader_election.cc:288] T 00000000000000000000000000000000 P 13bd21dfb6c24c24b9ca3519a7d860a5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:02:44.749154 28724 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 13bd21dfb6c24c24b9ca3519a7d860a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:02:44.749432 28724 raft_consensus.cc:686] T 00000000000000000000000000000000 P 13bd21dfb6c24c24b9ca3519a7d860a5 [term 1 LEADER]: Becoming Leader. State: Replica: 13bd21dfb6c24c24b9ca3519a7d860a5, State: Running, Role: LEADER
I20260811 19:02:44.749814 28724 consensus_queue.cc:237] T 00000000000000000000000000000000 P 13bd21dfb6c24c24b9ca3519a7d860a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13bd21dfb6c24c24b9ca3519a7d860a5" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 33739 } }
I20260811 19:02:44.750459 28717 sys_catalog.cc:564] T 00000000000000000000000000000000 P 13bd21dfb6c24c24b9ca3519a7d860a5 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:02:44.751662 28725 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13bd21dfb6c24c24b9ca3519a7d860a5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "13bd21dfb6c24c24b9ca3519a7d860a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13bd21dfb6c24c24b9ca3519a7d860a5" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 33739 } } }
I20260811 19:02:44.751811 28725 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13bd21dfb6c24c24b9ca3519a7d860a5 [sys.catalog]: This master's current role is: LEADER
I20260811 19:02:44.752100 28726 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13bd21dfb6c24c24b9ca3519a7d860a5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 13bd21dfb6c24c24b9ca3519a7d860a5. Latest consensus state: current_term: 1 leader_uuid: "13bd21dfb6c24c24b9ca3519a7d860a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13bd21dfb6c24c24b9ca3519a7d860a5" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 33739 } } }
I20260811 19:02:44.752211 28726 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13bd21dfb6c24c24b9ca3519a7d860a5 [sys.catalog]: This master's current role is: LEADER
I20260811 19:02:44.752331 28733 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:02:44.755366 28733 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:02:44.761075 28733 catalog_manager.cc:1347] Generated new cluster ID: 938ac96df5064bbe8ec60963fee370e8
I20260811 19:02:44.761157 28733 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:02:44.777087 28733 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:02:44.778542 28733 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:02:44.793286 28733 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 13bd21dfb6c24c24b9ca3519a7d860a5: Generated new TSK 0
I20260811 19:02:44.794318 28733 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 19:02:44.802115 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 28640
2026-08-11T19:02:44Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1529 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:02:44 dist-test-slave-0kls krb5kdc[28748](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:02:44 dist-test-slave-0kls krb5kdc[28748](info): set up 2 sockets
Aug 11 19:02:44 dist-test-slave-0kls krb5kdc[28748](info): commencing operation
krb5kdc: starting...
W20260811 19:02:45.910856 13748 mini_kdc.cc:121] Time spent starting KDC: real 1.085s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/krb5kdc/test-user.keytab.
Aug 11 19:02:45 dist-test-slave-0kls krb5kdc[28748](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474965, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:02:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:02:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.109.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.109.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.109.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab.
Entry for principal kudu/127.13.109.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab.
I20260811 19:02:46.049872 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.62:37073
--webserver_interface=127.13.109.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab
--principal=kudu/127.13.109.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:38669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.109.62:37073
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:02:46.170310 28761 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:02:46.170663 28761 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:02:46.170749 28761 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:02:46.174989 28761 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:02:46.175102 28761 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:02:46.175143 28761 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:02:46.175202 28761 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:02:46.175252 28761 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:02:46.180948 28761 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:38669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.109.62:37073
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab
--principal=kudu/127.13.109.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.62:37073
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.109.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28761
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:02:46.182379 28761 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:02:46.183655 28761 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:02:46.193908 28768 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:02:46.193908 28767 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:02:46.193958 28770 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:02:46.194151 28761 server_base.cc:1034] running on GCE node
I20260811 19:02:46.194900 28761 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:02:46.196156 28761 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:02:46.197335 28761 hybrid_clock.cc:648] HybridClock initialized: now 1786474966197310 us; error 75 us; skew 500 ppm
Aug 11 19:02:46 dist-test-slave-0kls krb5kdc[28748](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474966, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:02:46.200443 28761 init.cc:377] Logged in from keytab as kudu/127.13.109.62@KRBTEST.COM (short username kudu)
I20260811 19:02:46.201545 28761 webserver.cc:460] Webserver started at http://127.13.109.62:39633/ using document root <none> and password file <none>
I20260811 19:02:46.202286 28761 fs_manager.cc:359] Metadata directory not provided
I20260811 19:02:46.202399 28761 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:02:46.202677 28761 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:02:46.205039 28761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/master-0/data/instance:
uuid: "df03a25279794bc29ae9e20f2d28c530"
format_stamp: "Formatted at 2026-08-11 19:02:46 on dist-test-slave-0kls"
server_key: "f63b3ab253bfa165821434b618940ab1"
server_key_iv: "b4834185cf87588d47aea14f01621ed8"
server_key_version: "encryptionkey@0"
I20260811 19:02:46.205699 28761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/master-0/wal/instance:
uuid: "df03a25279794bc29ae9e20f2d28c530"
format_stamp: "Formatted at 2026-08-11 19:02:46 on dist-test-slave-0kls"
server_key: "f63b3ab253bfa165821434b618940ab1"
server_key_iv: "b4834185cf87588d47aea14f01621ed8"
server_key_version: "encryptionkey@0"
I20260811 19:02:46.210646 28761 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 19:02:46.213611 28777 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:46.215390 28761 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 19:02:46.215601 28761 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/master-0/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/master-0/wal
uuid: "df03a25279794bc29ae9e20f2d28c530"
format_stamp: "Formatted at 2026-08-11 19:02:46 on dist-test-slave-0kls"
server_key: "f63b3ab253bfa165821434b618940ab1"
server_key_iv: "b4834185cf87588d47aea14f01621ed8"
server_key_version: "encryptionkey@0"
I20260811 19:02:46.215770 28761 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:02:46.240329 28761 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:02:46.243556 28761 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:02:46.243782 28761 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:02:46.251701 28761 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.62:37073
I20260811 19:02:46.252007 28837 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.62:37073 every 8 connection(s)
I20260811 19:02:46.252928 28761 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
I20260811 19:02:46.256044 28838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:02:46.258018 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 28761
I20260811 19:02:46.258172 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/master-0/wal/instance
I20260811 19:02:46.258481 13748 external_mini_cluster.cc:1428] Setting key dc11109879958b4fa83e1e9c32be209b
I20260811 19:02:46.263900 28838 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df03a25279794bc29ae9e20f2d28c530: Bootstrap starting.
I20260811 19:02:46.267117 28838 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P df03a25279794bc29ae9e20f2d28c530: Neither blocks nor log segments found. Creating new log.
I20260811 19:02:46.267987 28838 log.cc:824] T 00000000000000000000000000000000 P df03a25279794bc29ae9e20f2d28c530: Log is configured to *not* fsync() on all Append() calls
Aug 11 19:02:46 dist-test-slave-0kls krb5kdc[28748](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474965, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
I20260811 19:02:46.270758 28838 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df03a25279794bc29ae9e20f2d28c530: No bootstrap required, opened a new log
I20260811 19:02:46.276134 28841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:46.260132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36120 (local address 127.13.109.62:37073)
0811 19:02:46.260647 (+ 515us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:46.260673 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:46.260720 (+ 47us) server_negotiation.cc:407] Connection header received
0811 19:02:46.261690 (+ 970us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:46.261720 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:46.262214 (+ 494us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:46.262642 (+ 428us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:02:46.263408 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:46.264903 (+ 1495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:46.265939 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:46.266636 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:46.266804 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:46.269483 (+ 2679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:02:46.269490 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:02:46.269508 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:02:46.269542 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:02:46.271587 (+ 2045us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:46.272132 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:46.272142 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:46.272150 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:46.272350 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:46.272867 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:46.272870 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:46.272871 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:46.273615 (+ 744us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:02:46.274577 (+ 962us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:46.274754 (+ 177us) server_negotiation.cc:299] Negotiation successful
0811 19:02:46.275171 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":138,"threads_started":1}
I20260811 19:02:46.276062 28838 raft_consensus.cc:348] T 00000000000000000000000000000000 P df03a25279794bc29ae9e20f2d28c530 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df03a25279794bc29ae9e20f2d28c530" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 37073 } }
I20260811 19:02:46.276376 28838 raft_consensus.cc:374] T 00000000000000000000000000000000 P df03a25279794bc29ae9e20f2d28c530 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:02:46.276439 28838 raft_consensus.cc:729] T 00000000000000000000000000000000 P df03a25279794bc29ae9e20f2d28c530 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df03a25279794bc29ae9e20f2d28c530, State: Initialized, Role: FOLLOWER
I20260811 19:02:46.276943 28838 consensus_queue.cc:260] T 00000000000000000000000000000000 P df03a25279794bc29ae9e20f2d28c530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df03a25279794bc29ae9e20f2d28c530" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 37073 } }
I20260811 19:02:46.277154 28838 raft_consensus.cc:388] T 00000000000000000000000000000000 P df03a25279794bc29ae9e20f2d28c530 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:02:46.277308 28838 raft_consensus.cc:482] T 00000000000000000000000000000000 P df03a25279794bc29ae9e20f2d28c530 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:02:46.277429 28838 raft_consensus.cc:3037] T 00000000000000000000000000000000 P df03a25279794bc29ae9e20f2d28c530 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:02:46.278800 28838 raft_consensus.cc:504] T 00000000000000000000000000000000 P df03a25279794bc29ae9e20f2d28c530 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df03a25279794bc29ae9e20f2d28c530" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 37073 } }
I20260811 19:02:46.279177 28838 leader_election.cc:302] T 00000000000000000000000000000000 P df03a25279794bc29ae9e20f2d28c530 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: df03a25279794bc29ae9e20f2d28c530; no voters:
I20260811 19:02:46.279469 28838 leader_election.cc:288] T 00000000000000000000000000000000 P df03a25279794bc29ae9e20f2d28c530 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:02:46.279618 28843 raft_consensus.cc:2781] T 00000000000000000000000000000000 P df03a25279794bc29ae9e20f2d28c530 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:02:46.279896 28843 raft_consensus.cc:686] T 00000000000000000000000000000000 P df03a25279794bc29ae9e20f2d28c530 [term 1 LEADER]: Becoming Leader. State: Replica: df03a25279794bc29ae9e20f2d28c530, State: Running, Role: LEADER
I20260811 19:02:46.280246 28843 consensus_queue.cc:237] T 00000000000000000000000000000000 P df03a25279794bc29ae9e20f2d28c530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df03a25279794bc29ae9e20f2d28c530" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 37073 } }
I20260811 19:02:46.280965 28838 sys_catalog.cc:564] T 00000000000000000000000000000000 P df03a25279794bc29ae9e20f2d28c530 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:02:46.281561 28843 sys_catalog.cc:455] T 00000000000000000000000000000000 P df03a25279794bc29ae9e20f2d28c530 [sys.catalog]: SysCatalogTable state changed. Reason: New leader df03a25279794bc29ae9e20f2d28c530. Latest consensus state: current_term: 1 leader_uuid: "df03a25279794bc29ae9e20f2d28c530" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df03a25279794bc29ae9e20f2d28c530" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 37073 } } }
I20260811 19:02:46.281701 28843 sys_catalog.cc:458] T 00000000000000000000000000000000 P df03a25279794bc29ae9e20f2d28c530 [sys.catalog]: This master's current role is: LEADER
I20260811 19:02:46.282262 28849 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:02:46.283535 28845 sys_catalog.cc:455] T 00000000000000000000000000000000 P df03a25279794bc29ae9e20f2d28c530 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "df03a25279794bc29ae9e20f2d28c530" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df03a25279794bc29ae9e20f2d28c530" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 37073 } } }
I20260811 19:02:46.283700 28845 sys_catalog.cc:458] T 00000000000000000000000000000000 P df03a25279794bc29ae9e20f2d28c530 [sys.catalog]: This master's current role is: LEADER
I20260811 19:02:46.285924 28849 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:02:46.292479 28849 catalog_manager.cc:1347] Generated new cluster ID: 77858b6acb72403bb610ad5d3895ea9a
I20260811 19:02:46.292573 28849 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:02:46.305809 28849 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:02:46.306916 28849 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:02:46.318470 28849 catalog_manager.cc:5868] T 00000000000000000000000000000000 P df03a25279794bc29ae9e20f2d28c530: Generated new TSK 0
I20260811 19:02:46.319537 28849 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.109.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.109.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.109.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.1.keytab.
Entry for principal kudu/127.13.109.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.1.keytab.
I20260811 19:02:46.366103 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.1:0
--local_ip_for_outbound_sockets=127.13.109.1
--webserver_interface=127.13.109.1
--webserver_port=0
--tserver_master_addrs=127.13.109.62:37073
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.1.keytab
--principal=kudu/127.13.109.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:38669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:02:46.490911 28866 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:02:46.491245 28866 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:02:46.491339 28866 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:02:46.495638 28866 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:02:46.495733 28866 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:02:46.495828 28866 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.1
I20260811 19:02:46.501439 28866 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:38669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.1.keytab
--principal=kudu/127.13.109.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.109.1
--webserver_port=0
--tserver_master_addrs=127.13.109.62:37073
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28866
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.1
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:02:46.502720 28866 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:02:46.503818 28866 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:02:46.514101 28872 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:02:46.514137 28871 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:02:46.514367 28866 server_base.cc:1034] running on GCE node
W20260811 19:02:46.514546 28874 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:02:46.514984 28866 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:02:46.515720 28866 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:02:46.516928 28866 hybrid_clock.cc:648] HybridClock initialized: now 1786474966516898 us; error 41 us; skew 500 ppm
Aug 11 19:02:46 dist-test-slave-0kls krb5kdc[28748](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474966, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:02:46.519942 28866 init.cc:377] Logged in from keytab as kudu/127.13.109.1@KRBTEST.COM (short username kudu)
I20260811 19:02:46.520953 28866 webserver.cc:460] Webserver started at http://127.13.109.1:46621/ using document root <none> and password file <none>
I20260811 19:02:46.521589 28866 fs_manager.cc:359] Metadata directory not provided
I20260811 19:02:46.521665 28866 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:02:46.521925 28866 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:02:46.524127 28866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-0/data/instance:
uuid: "a088e3d057b849d593f4d1972111e6c8"
format_stamp: "Formatted at 2026-08-11 19:02:46 on dist-test-slave-0kls"
server_key: "ef85a7a5af195b603801d7ede48c299e"
server_key_iv: "2dd632a0f7272b236484ff4c805f4ce1"
server_key_version: "encryptionkey@0"
I20260811 19:02:46.524717 28866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-0/wal/instance:
uuid: "a088e3d057b849d593f4d1972111e6c8"
format_stamp: "Formatted at 2026-08-11 19:02:46 on dist-test-slave-0kls"
server_key: "ef85a7a5af195b603801d7ede48c299e"
server_key_iv: "2dd632a0f7272b236484ff4c805f4ce1"
server_key_version: "encryptionkey@0"
I20260811 19:02:46.528960 28866 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.005s
I20260811 19:02:46.531917 28881 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:46.533578 28866 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 19:02:46.533725 28866 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-0/wal
uuid: "a088e3d057b849d593f4d1972111e6c8"
format_stamp: "Formatted at 2026-08-11 19:02:46 on dist-test-slave-0kls"
server_key: "ef85a7a5af195b603801d7ede48c299e"
server_key_iv: "2dd632a0f7272b236484ff4c805f4ce1"
server_key_version: "encryptionkey@0"
I20260811 19:02:46.533905 28866 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:02:46.550367 28866 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:02:46.553637 28866 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:02:46.553912 28866 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:02:46.554600 28866 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:02:46.555476 28866 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:02:46.555524 28866 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:46.555567 28866 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:02:46.555585 28866 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:46.565459 28866 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.1:42453
I20260811 19:02:46.565483 29015 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.1:42453 every 8 connection(s)
I20260811 19:02:46.566624 28866 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
I20260811 19:02:46.575131 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 28866
I20260811 19:02:46.575253 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-0/wal/instance
I20260811 19:02:46.575667 13748 external_mini_cluster.cc:1428] Setting key c5af8d8f8533714a122bfdc7cea603b4
Aug 11 19:02:46 dist-test-slave-0kls krb5kdc[28748](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474966, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.1@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
I20260811 19:02:46.582705 28841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:46.568675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.1:47661 (local address 127.13.109.62:37073)
0811 19:02:46.568830 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:46.568834 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:46.569713 (+ 879us) server_negotiation.cc:407] Connection header received
0811 19:02:46.571212 (+ 1499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:46.571215 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:46.571283 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:46.571370 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:02:46.572648 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:46.573201 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:46.574485 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:46.574741 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:46.574858 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:46.577972 (+ 3114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:02:46.577979 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:02:46.577985 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:02:46.578021 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:02:46.579826 (+ 1805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:46.580558 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:46.580561 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:46.580562 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:46.580638 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:46.581125 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:46.581128 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:46.581129 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:46.581387 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:02:46.581492 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:46.582252 (+ 760us) server_negotiation.cc:299] Negotiation successful
0811 19:02:46.582408 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 19:02:46.583616 29021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:46.569028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:37073 (local address 127.13.109.1:47661)
0811 19:02:46.569535 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:46.569583 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 19:02:46.570969 (+ 1386us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:02:46.571574 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:02:46.571582 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:02:46.571975 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:02:46.572482 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:46.572495 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:46.573345 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:46.573353 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:46.574344 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:46.574355 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:46.574925 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:46.574929 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:46.575112 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:46.575855 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:02:46.575871 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:02:46.577722 (+ 1851us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:02:46.579973 (+ 2251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:02:46.579982 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:02:46.580006 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:02:46.580443 (+ 437us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:02:46.580753 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:02:46.580758 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:02:46.580763 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:02:46.581012 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:02:46.581499 (+ 487us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:02:46.581505 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:02:46.581913 (+ 408us) client_negotiation.cc:769] Sending connection context
0811 19:02:46.582137 (+ 224us) client_negotiation.cc:240] Negotiation successful
0811 19:02:46.582492 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":123,"threads_started":1}
I20260811 19:02:46.585085 29017 heartbeater.cc:347] Connected to a master server at 127.13.109.62:37073
I20260811 19:02:46.585340 29017 heartbeater.cc:464] Registering TS with master...
I20260811 19:02:46.585965 29017 heartbeater.cc:511] Master 127.13.109.62:37073 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.13.109.2@KRBTEST.COM; defaulting to no policy
I20260811 19:02:46.587671 28794 ts_manager.cc:194] Registered new tserver with Master: a088e3d057b849d593f4d1972111e6c8 (127.13.109.1:42453)
I20260811 19:02:46.588954 28794 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.109.1@KRBTEST.COM'} at 127.13.109.1:47661
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.109.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.109.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.2.keytab.
Entry for principal kudu/127.13.109.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.2.keytab.
I20260811 19:02:46.607452 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.2:0
--local_ip_for_outbound_sockets=127.13.109.2
--webserver_interface=127.13.109.2
--webserver_port=0
--tserver_master_addrs=127.13.109.62:37073
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.2.keytab
--principal=kudu/127.13.109.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:38669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:02:46.726286 29024 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:02:46.726600 29024 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:02:46.726646 29024 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:02:46.731087 29024 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:02:46.731201 29024 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:02:46.731267 29024 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.2
I20260811 19:02:46.737114 29024 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:38669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.2.keytab
--principal=kudu/127.13.109.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.109.2
--webserver_port=0
--tserver_master_addrs=127.13.109.62:37073
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29024
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.2
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:02:46.738510 29024 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:02:46.739589 29024 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:02:46.749920 29034 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:02:46.749925 29031 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:02:46.750039 29024 server_base.cc:1034] running on GCE node
W20260811 19:02:46.749925 29030 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:02:46.750638 29024 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:02:46.751288 29024 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:02:46.752455 29024 hybrid_clock.cc:648] HybridClock initialized: now 1786474966752439 us; error 34 us; skew 500 ppm
Aug 11 19:02:46 dist-test-slave-0kls krb5kdc[28748](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474966, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:02:46.755458 29024 init.cc:377] Logged in from keytab as kudu/127.13.109.2@KRBTEST.COM (short username kudu)
I20260811 19:02:46.756392 29024 webserver.cc:460] Webserver started at http://127.13.109.2:38343/ using document root <none> and password file <none>
I20260811 19:02:46.756997 29024 fs_manager.cc:359] Metadata directory not provided
I20260811 19:02:46.757092 29024 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:02:46.757305 29024 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:02:46.759496 29024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-1/data/instance:
uuid: "fddff96ae4b84cdc86942277531be082"
format_stamp: "Formatted at 2026-08-11 19:02:46 on dist-test-slave-0kls"
server_key: "75888febf87c85be2459138e377876ca"
server_key_iv: "d05678c0d1c93a01a233a1475e19d331"
server_key_version: "encryptionkey@0"
I20260811 19:02:46.760115 29024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-1/wal/instance:
uuid: "fddff96ae4b84cdc86942277531be082"
format_stamp: "Formatted at 2026-08-11 19:02:46 on dist-test-slave-0kls"
server_key: "75888febf87c85be2459138e377876ca"
server_key_iv: "d05678c0d1c93a01a233a1475e19d331"
server_key_version: "encryptionkey@0"
I20260811 19:02:46.764067 29024 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 19:02:46.766927 29041 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:46.768200 29024 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 19:02:46.768356 29024 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-1/wal
uuid: "fddff96ae4b84cdc86942277531be082"
format_stamp: "Formatted at 2026-08-11 19:02:46 on dist-test-slave-0kls"
server_key: "75888febf87c85be2459138e377876ca"
server_key_iv: "d05678c0d1c93a01a233a1475e19d331"
server_key_version: "encryptionkey@0"
I20260811 19:02:46.768489 29024 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:02:46.805218 29024 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:02:46.809029 29024 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:02:46.809273 29024 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:02:46.810058 29024 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:02:46.811012 29024 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:02:46.811094 29024 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:46.811169 29024 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:02:46.811232 29024 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:46.821398 29024 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.2:34823
I20260811 19:02:46.821442 29167 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.2:34823 every 8 connection(s)
I20260811 19:02:46.822466 29024 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
I20260811 19:02:46.825827 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 29024
I20260811 19:02:46.825958 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-1/wal/instance
I20260811 19:02:46.826238 13748 external_mini_cluster.cc:1428] Setting key 5fa2a5c1d256af940e7339a41d525ce0
Aug 11 19:02:46 dist-test-slave-0kls krb5kdc[28748](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474966, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.2@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.109.3@KRBTEST.COM; defaulting to no policy
I20260811 19:02:46.839166 28841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:46.824462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.2:45989 (local address 127.13.109.62:37073)
0811 19:02:46.824634 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:46.824639 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:46.825519 (+ 880us) server_negotiation.cc:407] Connection header received
0811 19:02:46.827359 (+ 1840us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:46.827362 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:46.827409 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:46.827484 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:02:46.828844 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:46.829682 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:46.831585 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:46.831810 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:46.831896 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:46.835112 (+ 3216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:02:46.835115 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:02:46.835117 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:02:46.835147 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:02:46.836701 (+ 1554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:46.837318 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:46.837321 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:46.837323 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:46.837406 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:46.837891 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:46.837894 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:46.837896 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:46.838128 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:02:46.838248 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:46.838835 (+ 587us) server_negotiation.cc:299] Negotiation successful
0811 19:02:46.838973 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 19:02:46.840222 29173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:46.824852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:37073 (local address 127.13.109.2:45989)
0811 19:02:46.825353 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:46.825399 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 19:02:46.827127 (+ 1728us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:02:46.827698 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:02:46.827708 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:02:46.828138 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:02:46.828675 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:46.828690 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:46.829894 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:46.829901 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:46.831431 (+ 1530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:46.831448 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:46.832014 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:46.832020 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:46.832274 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:46.832972 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:02:46.833002 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:02:46.834898 (+ 1896us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:02:46.836847 (+ 1949us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:02:46.836853 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:02:46.836874 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:02:46.837188 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:02:46.837523 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:02:46.837526 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:02:46.837528 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:02:46.837752 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:02:46.838247 (+ 495us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:02:46.838257 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:02:46.838549 (+ 292us) client_negotiation.cc:769] Sending connection context
0811 19:02:46.838777 (+ 228us) client_negotiation.cc:240] Negotiation successful
0811 19:02:46.839153 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":120,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.109.3@KRBTEST.COM" created.
I20260811 19:02:46.841678 29169 heartbeater.cc:347] Connected to a master server at 127.13.109.62:37073
I20260811 19:02:46.842026 29169 heartbeater.cc:464] Registering TS with master...
I20260811 19:02:46.842733 29169 heartbeater.cc:511] Master 127.13.109.62:37073 requested a full tablet report, sending...
I20260811 19:02:46.844224 28794 ts_manager.cc:194] Registered new tserver with Master: fddff96ae4b84cdc86942277531be082 (127.13.109.2:34823)
I20260811 19:02:46.845052 28794 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.109.2@KRBTEST.COM'} at 127.13.109.2:45989
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.109.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.3.keytab.
Entry for principal kudu/127.13.109.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.3.keytab.
I20260811 19:02:46.861115 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.3:0
--local_ip_for_outbound_sockets=127.13.109.3
--webserver_interface=127.13.109.3
--webserver_port=0
--tserver_master_addrs=127.13.109.62:37073
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.3.keytab
--principal=kudu/127.13.109.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:38669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:02:46.985582 29177 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:02:46.985996 29177 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:02:46.986095 29177 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:02:46.990376 29177 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:02:46.990468 29177 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:02:46.990569 29177 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.3
I20260811 19:02:46.996289 29177 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:38669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.3.keytab
--principal=kudu/127.13.109.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.109.3
--webserver_port=0
--tserver_master_addrs=127.13.109.62:37073
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29177
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.3
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:02:46.997666 29177 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:02:46.998832 29177 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:02:47.010200 29182 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:02:47.010190 29183 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:02:47.010190 29185 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:02:47.010650 29177 server_base.cc:1034] running on GCE node
I20260811 19:02:47.011142 29177 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:02:47.011888 29177 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:02:47.013137 29177 hybrid_clock.cc:648] HybridClock initialized: now 1786474967013113 us; error 35 us; skew 500 ppm
Aug 11 19:02:47 dist-test-slave-0kls krb5kdc[28748](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474967, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:02:47.016975 29177 init.cc:377] Logged in from keytab as kudu/127.13.109.3@KRBTEST.COM (short username kudu)
I20260811 19:02:47.018234 29177 webserver.cc:460] Webserver started at http://127.13.109.3:39733/ using document root <none> and password file <none>
I20260811 19:02:47.018900 29177 fs_manager.cc:359] Metadata directory not provided
I20260811 19:02:47.019006 29177 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:02:47.019279 29177 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:02:47.021742 29177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-2/data/instance:
uuid: "7f911943ce2449aba06ac11215b59788"
format_stamp: "Formatted at 2026-08-11 19:02:47 on dist-test-slave-0kls"
server_key: "b58775d7cbbb98b1c1833ff3b12d2d18"
server_key_iv: "72f02612bbefbb73059d05dc5554a9f8"
server_key_version: "encryptionkey@0"
I20260811 19:02:47.022428 29177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-2/wal/instance:
uuid: "7f911943ce2449aba06ac11215b59788"
format_stamp: "Formatted at 2026-08-11 19:02:47 on dist-test-slave-0kls"
server_key: "b58775d7cbbb98b1c1833ff3b12d2d18"
server_key_iv: "72f02612bbefbb73059d05dc5554a9f8"
server_key_version: "encryptionkey@0"
I20260811 19:02:47.026814 29177 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 19:02:47.029510 29192 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:47.030896 29177 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 19:02:47.031051 29177 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-2/wal
uuid: "7f911943ce2449aba06ac11215b59788"
format_stamp: "Formatted at 2026-08-11 19:02:47 on dist-test-slave-0kls"
server_key: "b58775d7cbbb98b1c1833ff3b12d2d18"
server_key_iv: "72f02612bbefbb73059d05dc5554a9f8"
server_key_version: "encryptionkey@0"
I20260811 19:02:47.031189 29177 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:02:47.049547 29177 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:02:47.053613 29177 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:02:47.053915 29177 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:02:47.054662 29177 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:02:47.055676 29177 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:02:47.055770 29177 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:47.055891 29177 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:02:47.055948 29177 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:47.066743 29177 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.3:43983
I20260811 19:02:47.066764 29318 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.3:43983 every 8 connection(s)
I20260811 19:02:47.067986 29177 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
I20260811 19:02:47.069662 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 29177
I20260811 19:02:47.069837 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-2/wal/instance
I20260811 19:02:47.070200 13748 external_mini_cluster.cc:1428] Setting key 9fad5ffde191b29beba915d99b070732
Aug 11 19:02:47 dist-test-slave-0kls krb5kdc[28748](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474967, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.3@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
I20260811 19:02:47.084811 28841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:47.072063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.3:45609 (local address 127.13.109.62:37073)
0811 19:02:47.072209 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:47.072214 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:47.072229 (+ 15us) server_negotiation.cc:407] Connection header received
0811 19:02:47.073716 (+ 1487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:47.073719 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:47.073772 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:47.073910 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:02:47.075219 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:47.075806 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:47.077002 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:47.077264 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:47.077356 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:47.080129 (+ 2773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:02:47.080134 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:02:47.080136 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:02:47.080168 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:02:47.082107 (+ 1939us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:47.082834 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:47.082837 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:47.082838 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:47.082933 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:47.083329 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:47.083333 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:47.083334 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:47.083630 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:02:47.083737 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:47.084469 (+ 732us) server_negotiation.cc:299] Negotiation successful
0811 19:02:47.084631 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 19:02:47.085938 29322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:47.070823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:37073 (local address 127.13.109.3:45609)
0811 19:02:47.071527 (+ 704us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:47.071575 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 19:02:47.073493 (+ 1918us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:02:47.074100 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:02:47.074107 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:02:47.074523 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:02:47.075053 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:47.075073 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:47.075959 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:47.075962 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:47.076872 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:47.076882 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:47.077417 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:47.077419 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:47.077575 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:47.078264 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:02:47.078282 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:02:47.079883 (+ 1601us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:02:47.082318 (+ 2435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:02:47.082325 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:02:47.082344 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:02:47.082666 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:02:47.083047 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:02:47.083049 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:02:47.083051 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:02:47.083219 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:02:47.083761 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:02:47.083769 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:02:47.084131 (+ 362us) client_negotiation.cc:769] Sending connection context
0811 19:02:47.084414 (+ 283us) client_negotiation.cc:240] Negotiation successful
0811 19:02:47.084802 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":177,"threads_started":1}
I20260811 19:02:47.087126 29320 heartbeater.cc:347] Connected to a master server at 127.13.109.62:37073
I20260811 19:02:47.087360 29320 heartbeater.cc:464] Registering TS with master...
I20260811 19:02:47.087855 29320 heartbeater.cc:511] Master 127.13.109.62:37073 requested a full tablet report, sending...
I20260811 19:02:47.088923 28794 ts_manager.cc:194] Registered new tserver with Master: 7f911943ce2449aba06ac11215b59788 (127.13.109.3:43983)
I20260811 19:02:47.089519 28794 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.109.3@KRBTEST.COM'} at 127.13.109.3:45609
I20260811 19:02:47.097884 13748 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 19:02:47.108542 28841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:47.100184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36122 (local address 127.13.109.62:37073)
0811 19:02:47.100331 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:47.100336 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:47.100603 (+ 267us) server_negotiation.cc:407] Connection header received
0811 19:02:47.100724 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:47.100727 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:47.100772 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:47.100842 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:02:47.101648 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:47.102258 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:47.103127 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:47.103334 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:47.103409 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:47.104849 (+ 1440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:02:47.104852 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:02:47.104854 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:02:47.104879 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:02:47.106364 (+ 1485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:47.106813 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:47.106816 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:47.106817 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:47.106894 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:47.107306 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:47.107311 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:47.107314 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:47.107696 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:02:47.107833 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:47.108196 (+ 363us) server_negotiation.cc:299] Negotiation successful
0811 19:02:47.108326 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 19:02:47.111720 28794 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36122:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 19:02:47.114921 28794 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 19:02:47.130719 29336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:47.125564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39998 (local address 127.13.109.1:42453)
0811 19:02:47.126015 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:47.126022 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:47.126357 (+ 335us) server_negotiation.cc:407] Connection header received
0811 19:02:47.126444 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:47.126449 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:47.126635 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:47.126768 (+ 133us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:02:47.127024 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:47.128123 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:47.129331 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:47.130018 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:47.130144 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:47.130311 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:47.130455 (+ 144us) server_negotiation.cc:299] Negotiation successful
0811 19:02:47.130555 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":115,"threads_started":1}
I20260811 19:02:47.130712 29334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:47.125165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.1:42453 (local address 127.0.0.1:39998)
0811 19:02:47.126133 (+ 968us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:47.126153 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 19:02:47.126255 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:02:47.126771 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:02:47.126774 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:02:47.126793 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:02:47.126887 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:47.126893 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:47.128226 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:47.128232 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:47.129162 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:47.129172 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:47.130144 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:47.130148 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:47.130271 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:47.130311 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 19:02:47.130426 (+ 115us) client_negotiation.cc:240] Negotiation successful
0811 19:02:47.130555 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":905,"mutex_wait_us":4,"thread_start_us":95,"threads_started":1}
I20260811 19:02:47.131012 29332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:47.124361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.2:34823 (local address 127.0.0.1:50900)
0811 19:02:47.124971 (+ 610us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:47.125032 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 19:02:47.125172 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:02:47.126246 (+ 1074us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:02:47.126252 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:02:47.126275 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:02:47.126433 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:47.126443 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:47.127629 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:47.127632 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:47.128833 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:47.128843 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:47.130430 (+ 1587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:47.130434 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:47.130621 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:47.130701 (+ 80us) client_negotiation.cc:769] Sending connection context
0811 19:02:47.130774 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 19:02:47.130865 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":406,"mutex_wait_us":64,"thread_start_us":106,"threads_started":1}
I20260811 19:02:47.131373 29331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:47.124361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.3:43983 (local address 127.0.0.1:54892)
0811 19:02:47.125082 (+ 721us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:47.125096 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 19:02:47.125177 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:02:47.126393 (+ 1216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:02:47.126396 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:02:47.126419 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:02:47.126529 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:47.126535 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:47.128178 (+ 1643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:47.128181 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:47.129243 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:47.129252 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:47.130444 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:47.130447 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:47.130586 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:47.130703 (+ 117us) client_negotiation.cc:769] Sending connection context
0811 19:02:47.130774 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 19:02:47.131283 (+ 509us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":649,"mutex_wait_us":261,"thread_start_us":105,"threads_started":1}
I20260811 19:02:47.131476 29333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:47.124665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50900 (local address 127.13.109.2:34823)
0811 19:02:47.125234 (+ 569us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:47.125254 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:47.125281 (+ 27us) server_negotiation.cc:407] Connection header received
0811 19:02:47.125367 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:47.125372 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:47.125619 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:47.125783 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:02:47.126572 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:47.127496 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:47.128973 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:47.130093 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:47.130203 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:47.130361 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:47.131238 (+ 877us) server_negotiation.cc:299] Negotiation successful
0811 19:02:47.131338 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":454,"thread_start_us":113,"threads_started":1}
I20260811 19:02:47.131910 29335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:47.125559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54892 (local address 127.13.109.3:43983)
0811 19:02:47.125912 (+ 353us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:47.125925 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:47.125947 (+ 22us) server_negotiation.cc:407] Connection header received
0811 19:02:47.126025 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:47.126029 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:47.126236 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:47.126693 (+ 457us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:02:47.126854 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:47.128028 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:47.129379 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:47.130192 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:47.130281 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:47.130394 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:47.131679 (+ 1285us) server_negotiation.cc:299] Negotiation successful
0811 19:02:47.131777 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":250,"thread_start_us":145,"threads_started":1}
I20260811 19:02:47.134074 29094 tablet_service.cc:1467] Processing CreateTablet for tablet 450ad0b4c88b4950959fd49708969f02 (DEFAULT_TABLE table=test-table [id=af518d31ad664bd6bc5d2db566af3237]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:02:47.134083 28939 tablet_service.cc:1467] Processing CreateTablet for tablet 450ad0b4c88b4950959fd49708969f02 (DEFAULT_TABLE table=test-table [id=af518d31ad664bd6bc5d2db566af3237]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:02:47.134549 29244 tablet_service.cc:1467] Processing CreateTablet for tablet 450ad0b4c88b4950959fd49708969f02 (DEFAULT_TABLE table=test-table [id=af518d31ad664bd6bc5d2db566af3237]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:02:47.135214 29094 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 450ad0b4c88b4950959fd49708969f02. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:02:47.135272 28939 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 450ad0b4c88b4950959fd49708969f02. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:02:47.135505 29244 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 450ad0b4c88b4950959fd49708969f02. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:02:47.141992 29337 tablet_bootstrap.cc:492] T 450ad0b4c88b4950959fd49708969f02 P fddff96ae4b84cdc86942277531be082: Bootstrap starting.
I20260811 19:02:47.143219 29338 tablet_bootstrap.cc:492] T 450ad0b4c88b4950959fd49708969f02 P a088e3d057b849d593f4d1972111e6c8: Bootstrap starting.
I20260811 19:02:47.143510 29339 tablet_bootstrap.cc:492] T 450ad0b4c88b4950959fd49708969f02 P 7f911943ce2449aba06ac11215b59788: Bootstrap starting.
I20260811 19:02:47.145143 29338 tablet_bootstrap.cc:654] T 450ad0b4c88b4950959fd49708969f02 P a088e3d057b849d593f4d1972111e6c8: Neither blocks nor log segments found. Creating new log.
I20260811 19:02:47.145428 29337 tablet_bootstrap.cc:654] T 450ad0b4c88b4950959fd49708969f02 P fddff96ae4b84cdc86942277531be082: Neither blocks nor log segments found. Creating new log.
I20260811 19:02:47.146039 29338 log.cc:824] T 450ad0b4c88b4950959fd49708969f02 P a088e3d057b849d593f4d1972111e6c8: Log is configured to *not* fsync() on all Append() calls
I20260811 19:02:47.146081 29339 tablet_bootstrap.cc:654] T 450ad0b4c88b4950959fd49708969f02 P 7f911943ce2449aba06ac11215b59788: Neither blocks nor log segments found. Creating new log.
I20260811 19:02:47.146399 29337 log.cc:824] T 450ad0b4c88b4950959fd49708969f02 P fddff96ae4b84cdc86942277531be082: Log is configured to *not* fsync() on all Append() calls
I20260811 19:02:47.147140 29339 log.cc:824] T 450ad0b4c88b4950959fd49708969f02 P 7f911943ce2449aba06ac11215b59788: Log is configured to *not* fsync() on all Append() calls
I20260811 19:02:47.148351 29337 tablet_bootstrap.cc:492] T 450ad0b4c88b4950959fd49708969f02 P fddff96ae4b84cdc86942277531be082: No bootstrap required, opened a new log
I20260811 19:02:47.148345 29338 tablet_bootstrap.cc:492] T 450ad0b4c88b4950959fd49708969f02 P a088e3d057b849d593f4d1972111e6c8: No bootstrap required, opened a new log
I20260811 19:02:47.148581 29337 ts_tablet_manager.cc:1397] T 450ad0b4c88b4950959fd49708969f02 P fddff96ae4b84cdc86942277531be082: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 19:02:47.148582 29338 ts_tablet_manager.cc:1397] T 450ad0b4c88b4950959fd49708969f02 P a088e3d057b849d593f4d1972111e6c8: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260811 19:02:47.149115 29339 tablet_bootstrap.cc:492] T 450ad0b4c88b4950959fd49708969f02 P 7f911943ce2449aba06ac11215b59788: No bootstrap required, opened a new log
I20260811 19:02:47.149291 29339 ts_tablet_manager.cc:1397] T 450ad0b4c88b4950959fd49708969f02 P 7f911943ce2449aba06ac11215b59788: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 19:02:47.151958 29337 raft_consensus.cc:348] T 450ad0b4c88b4950959fd49708969f02 P fddff96ae4b84cdc86942277531be082 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f911943ce2449aba06ac11215b59788" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43983 } } peers { permanent_uuid: "fddff96ae4b84cdc86942277531be082" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34823 } } peers { permanent_uuid: "a088e3d057b849d593f4d1972111e6c8" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 42453 } }
I20260811 19:02:47.152274 29337 raft_consensus.cc:374] T 450ad0b4c88b4950959fd49708969f02 P fddff96ae4b84cdc86942277531be082 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:02:47.152343 29337 raft_consensus.cc:729] T 450ad0b4c88b4950959fd49708969f02 P fddff96ae4b84cdc86942277531be082 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fddff96ae4b84cdc86942277531be082, State: Initialized, Role: FOLLOWER
I20260811 19:02:47.152902 29337 consensus_queue.cc:260] T 450ad0b4c88b4950959fd49708969f02 P fddff96ae4b84cdc86942277531be082 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f911943ce2449aba06ac11215b59788" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43983 } } peers { permanent_uuid: "fddff96ae4b84cdc86942277531be082" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34823 } } peers { permanent_uuid: "a088e3d057b849d593f4d1972111e6c8" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 42453 } }
I20260811 19:02:47.153340 29338 raft_consensus.cc:348] T 450ad0b4c88b4950959fd49708969f02 P a088e3d057b849d593f4d1972111e6c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f911943ce2449aba06ac11215b59788" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43983 } } peers { permanent_uuid: "fddff96ae4b84cdc86942277531be082" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34823 } } peers { permanent_uuid: "a088e3d057b849d593f4d1972111e6c8" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 42453 } }
I20260811 19:02:47.153631 29338 raft_consensus.cc:374] T 450ad0b4c88b4950959fd49708969f02 P a088e3d057b849d593f4d1972111e6c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:02:47.153690 29338 raft_consensus.cc:729] T 450ad0b4c88b4950959fd49708969f02 P a088e3d057b849d593f4d1972111e6c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a088e3d057b849d593f4d1972111e6c8, State: Initialized, Role: FOLLOWER
I20260811 19:02:47.153733 29169 heartbeater.cc:503] Master 127.13.109.62:37073 was elected leader, sending a full tablet report...
I20260811 19:02:47.154137 29337 ts_tablet_manager.cc:1428] T 450ad0b4c88b4950959fd49708969f02 P fddff96ae4b84cdc86942277531be082: Time spent starting tablet: real 0.005s user 0.007s sys 0.000s
I20260811 19:02:47.154170 29338 consensus_queue.cc:260] T 450ad0b4c88b4950959fd49708969f02 P a088e3d057b849d593f4d1972111e6c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f911943ce2449aba06ac11215b59788" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43983 } } peers { permanent_uuid: "fddff96ae4b84cdc86942277531be082" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34823 } } peers { permanent_uuid: "a088e3d057b849d593f4d1972111e6c8" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 42453 } }
I20260811 19:02:47.154902 29017 heartbeater.cc:503] Master 127.13.109.62:37073 was elected leader, sending a full tablet report...
I20260811 19:02:47.155179 29338 ts_tablet_manager.cc:1428] T 450ad0b4c88b4950959fd49708969f02 P a088e3d057b849d593f4d1972111e6c8: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260811 19:02:47.153409 29339 raft_consensus.cc:348] T 450ad0b4c88b4950959fd49708969f02 P 7f911943ce2449aba06ac11215b59788 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f911943ce2449aba06ac11215b59788" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43983 } } peers { permanent_uuid: "fddff96ae4b84cdc86942277531be082" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34823 } } peers { permanent_uuid: "a088e3d057b849d593f4d1972111e6c8" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 42453 } }
I20260811 19:02:47.155575 29339 raft_consensus.cc:374] T 450ad0b4c88b4950959fd49708969f02 P 7f911943ce2449aba06ac11215b59788 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:02:47.155637 29339 raft_consensus.cc:729] T 450ad0b4c88b4950959fd49708969f02 P 7f911943ce2449aba06ac11215b59788 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f911943ce2449aba06ac11215b59788, State: Initialized, Role: FOLLOWER
I20260811 19:02:47.156093 29339 consensus_queue.cc:260] T 450ad0b4c88b4950959fd49708969f02 P 7f911943ce2449aba06ac11215b59788 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f911943ce2449aba06ac11215b59788" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43983 } } peers { permanent_uuid: "fddff96ae4b84cdc86942277531be082" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34823 } } peers { permanent_uuid: "a088e3d057b849d593f4d1972111e6c8" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 42453 } }
I20260811 19:02:47.157166 29339 ts_tablet_manager.cc:1428] T 450ad0b4c88b4950959fd49708969f02 P 7f911943ce2449aba06ac11215b59788: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260811 19:02:47.157166 29320 heartbeater.cc:503] Master 127.13.109.62:37073 was elected leader, sending a full tablet report...
I20260811 19:02:47.278869 29345 raft_consensus.cc:482] T 450ad0b4c88b4950959fd49708969f02 P a088e3d057b849d593f4d1972111e6c8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:02:47.279141 29345 raft_consensus.cc:504] T 450ad0b4c88b4950959fd49708969f02 P a088e3d057b849d593f4d1972111e6c8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f911943ce2449aba06ac11215b59788" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43983 } } peers { permanent_uuid: "fddff96ae4b84cdc86942277531be082" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34823 } } peers { permanent_uuid: "a088e3d057b849d593f4d1972111e6c8" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 42453 } }
I20260811 19:02:47.280416 29345 leader_election.cc:288] T 450ad0b4c88b4950959fd49708969f02 P a088e3d057b849d593f4d1972111e6c8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7f911943ce2449aba06ac11215b59788 (127.13.109.3:43983), fddff96ae4b84cdc86942277531be082 (127.13.109.2:34823)
I20260811 19:02:47.284996 29333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:47.281216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.1:59571 (local address 127.13.109.2:34823)
0811 19:02:47.281364 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:47.281368 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:47.281380 (+ 12us) server_negotiation.cc:407] Connection header received
0811 19:02:47.281427 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:47.281430 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:47.281471 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:47.281562 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:02:47.282110 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:47.282666 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:47.284000 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:47.284404 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:47.284536 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:47.284593 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:47.284785 (+ 192us) server_negotiation.cc:299] Negotiation successful
0811 19:02:47.284857 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 19:02:47.285019 29335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:47.281098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.1:60307 (local address 127.13.109.3:43983)
0811 19:02:47.281365 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:47.281369 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:47.281381 (+ 12us) server_negotiation.cc:407] Connection header received
0811 19:02:47.281428 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:47.281430 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:47.281468 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:47.281563 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:02:47.282083 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:47.282704 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:47.284012 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:47.284426 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:47.284562 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:47.284618 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:47.284821 (+ 203us) server_negotiation.cc:299] Negotiation successful
0811 19:02:47.284890 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187}
I20260811 19:02:47.285053 29347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:47.280650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.3:43983 (local address 127.13.109.1:60307)
0811 19:02:47.280983 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:47.281004 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 19:02:47.281126 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:02:47.281581 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:02:47.281583 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:02:47.281620 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:02:47.281939 (+ 319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:47.281946 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:47.282820 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:47.282824 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:47.283884 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:47.283894 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:47.284562 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:47.284565 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:47.284689 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:47.284715 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 19:02:47.284803 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 19:02:47.284878 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"mutex_wait_us":95,"spinlock_wait_cycles":24704,"thread_start_us":104,"threads_started":1}
I20260811 19:02:47.285060 29348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:47.280696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.2:34823 (local address 127.13.109.1:59571)
0811 19:02:47.281048 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:47.281063 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 19:02:47.281167 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:02:47.281577 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:02:47.281580 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:02:47.281609 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:02:47.281946 (+ 337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:47.281952 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:47.282783 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:47.282787 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:47.283871 (+ 1084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:47.283882 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:47.284536 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:47.284540 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:47.284640 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:47.284672 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 19:02:47.284767 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 19:02:47.284842 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"mutex_wait_us":58,"spinlock_wait_cycles":2816,"thread_start_us":147,"threads_started":1}
I20260811 19:02:47.286002 29117 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "450ad0b4c88b4950959fd49708969f02" candidate_uuid: "a088e3d057b849d593f4d1972111e6c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fddff96ae4b84cdc86942277531be082" is_pre_election: true
I20260811 19:02:47.286375 29117 raft_consensus.cc:2454] T 450ad0b4c88b4950959fd49708969f02 P fddff96ae4b84cdc86942277531be082 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a088e3d057b849d593f4d1972111e6c8 in term 0.
I20260811 19:02:47.286891 29265 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "450ad0b4c88b4950959fd49708969f02" candidate_uuid: "a088e3d057b849d593f4d1972111e6c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f911943ce2449aba06ac11215b59788" is_pre_election: true
I20260811 19:02:47.286919 28882 leader_election.cc:302] T 450ad0b4c88b4950959fd49708969f02 P a088e3d057b849d593f4d1972111e6c8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a088e3d057b849d593f4d1972111e6c8, fddff96ae4b84cdc86942277531be082; no voters:
I20260811 19:02:47.287179 29265 raft_consensus.cc:2454] T 450ad0b4c88b4950959fd49708969f02 P 7f911943ce2449aba06ac11215b59788 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a088e3d057b849d593f4d1972111e6c8 in term 0.
I20260811 19:02:47.287253 29345 raft_consensus.cc:2781] T 450ad0b4c88b4950959fd49708969f02 P a088e3d057b849d593f4d1972111e6c8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 19:02:47.287382 29345 raft_consensus.cc:482] T 450ad0b4c88b4950959fd49708969f02 P a088e3d057b849d593f4d1972111e6c8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 19:02:47.287456 29345 raft_consensus.cc:3037] T 450ad0b4c88b4950959fd49708969f02 P a088e3d057b849d593f4d1972111e6c8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:02:47.288622 29345 raft_consensus.cc:504] T 450ad0b4c88b4950959fd49708969f02 P a088e3d057b849d593f4d1972111e6c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f911943ce2449aba06ac11215b59788" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43983 } } peers { permanent_uuid: "fddff96ae4b84cdc86942277531be082" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34823 } } peers { permanent_uuid: "a088e3d057b849d593f4d1972111e6c8" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 42453 } }
I20260811 19:02:47.289060 29345 leader_election.cc:288] T 450ad0b4c88b4950959fd49708969f02 P a088e3d057b849d593f4d1972111e6c8 [CANDIDATE]: Term 1 election: Requested vote from peers 7f911943ce2449aba06ac11215b59788 (127.13.109.3:43983), fddff96ae4b84cdc86942277531be082 (127.13.109.2:34823)
I20260811 19:02:47.289474 29265 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "450ad0b4c88b4950959fd49708969f02" candidate_uuid: "a088e3d057b849d593f4d1972111e6c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f911943ce2449aba06ac11215b59788"
I20260811 19:02:47.289592 29265 raft_consensus.cc:3037] T 450ad0b4c88b4950959fd49708969f02 P 7f911943ce2449aba06ac11215b59788 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:02:47.289963 29117 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "450ad0b4c88b4950959fd49708969f02" candidate_uuid: "a088e3d057b849d593f4d1972111e6c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fddff96ae4b84cdc86942277531be082"
I20260811 19:02:47.290115 29117 raft_consensus.cc:3037] T 450ad0b4c88b4950959fd49708969f02 P fddff96ae4b84cdc86942277531be082 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:02:47.290908 29265 raft_consensus.cc:2454] T 450ad0b4c88b4950959fd49708969f02 P 7f911943ce2449aba06ac11215b59788 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a088e3d057b849d593f4d1972111e6c8 in term 1.
I20260811 19:02:47.291450 29117 raft_consensus.cc:2454] T 450ad0b4c88b4950959fd49708969f02 P fddff96ae4b84cdc86942277531be082 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a088e3d057b849d593f4d1972111e6c8 in term 1.
I20260811 19:02:47.291848 28882 leader_election.cc:302] T 450ad0b4c88b4950959fd49708969f02 P a088e3d057b849d593f4d1972111e6c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a088e3d057b849d593f4d1972111e6c8, fddff96ae4b84cdc86942277531be082; no voters:
I20260811 19:02:47.292140 29345 raft_consensus.cc:2781] T 450ad0b4c88b4950959fd49708969f02 P a088e3d057b849d593f4d1972111e6c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:02:47.292394 29345 raft_consensus.cc:686] T 450ad0b4c88b4950959fd49708969f02 P a088e3d057b849d593f4d1972111e6c8 [term 1 LEADER]: Becoming Leader. State: Replica: a088e3d057b849d593f4d1972111e6c8, State: Running, Role: LEADER
I20260811 19:02:47.292804 29345 consensus_queue.cc:237] T 450ad0b4c88b4950959fd49708969f02 P a088e3d057b849d593f4d1972111e6c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f911943ce2449aba06ac11215b59788" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43983 } } peers { permanent_uuid: "fddff96ae4b84cdc86942277531be082" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34823 } } peers { permanent_uuid: "a088e3d057b849d593f4d1972111e6c8" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 42453 } }
I20260811 19:02:47.296605 28793 catalog_manager.cc:5495] T 450ad0b4c88b4950959fd49708969f02 P a088e3d057b849d593f4d1972111e6c8 reported cstate change: term changed from 0 to 1, leader changed from <none> to a088e3d057b849d593f4d1972111e6c8 (127.13.109.1). New cstate: current_term: 1 leader_uuid: "a088e3d057b849d593f4d1972111e6c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f911943ce2449aba06ac11215b59788" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fddff96ae4b84cdc86942277531be082" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34823 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a088e3d057b849d593f4d1972111e6c8" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 42453 } health_report { overall_health: HEALTHY } } }
W20260811 19:02:47.318185 29018 tablet.cc:2369] T 450ad0b4c88b4950959fd49708969f02 P a088e3d057b849d593f4d1972111e6c8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 19:02:47.318975 29321 tablet.cc:2369] T 450ad0b4c88b4950959fd49708969f02 P 7f911943ce2449aba06ac11215b59788: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 19:02:47.323726 29172 tablet.cc:2369] T 450ad0b4c88b4950959fd49708969f02 P fddff96ae4b84cdc86942277531be082: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 19:02:47.325901 29336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:47.322258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40006 (local address 127.13.109.1:42453)
0811 19:02:47.322397 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:47.322401 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:47.322415 (+ 14us) server_negotiation.cc:407] Connection header received
0811 19:02:47.322500 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:47.322503 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:47.322558 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:47.322648 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:02:47.322886 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:47.323390 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:47.324352 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:47.324578 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:47.324672 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:47.324955 (+ 283us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:02:47.325392 (+ 437us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:02:47.325482 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:47.325617 (+ 135us) server_negotiation.cc:299] Negotiation successful
0811 19:02:47.325686 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 19:02:47.332103 29265 raft_consensus.cc:1261] T 450ad0b4c88b4950959fd49708969f02 P 7f911943ce2449aba06ac11215b59788 [term 1 FOLLOWER]: Refusing update from remote peer a088e3d057b849d593f4d1972111e6c8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:02:47.332103 29117 raft_consensus.cc:1261] T 450ad0b4c88b4950959fd49708969f02 P fddff96ae4b84cdc86942277531be082 [term 1 FOLLOWER]: Refusing update from remote peer a088e3d057b849d593f4d1972111e6c8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:02:47.332918 29345 consensus_queue.cc:1035] T 450ad0b4c88b4950959fd49708969f02 P a088e3d057b849d593f4d1972111e6c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fddff96ae4b84cdc86942277531be082" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34823 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:02:47.333168 29349 consensus_queue.cc:1035] T 450ad0b4c88b4950959fd49708969f02 P a088e3d057b849d593f4d1972111e6c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f911943ce2449aba06ac11215b59788" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:02:47.350507 28793 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36122:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:02:47.350729 28793 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36122:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:02:47.353518 28793 catalog_manager.cc:5782] T 00000000000000000000000000000000 P df03a25279794bc29ae9e20f2d28c530: Sending DeleteTablet for 3 replicas of tablet 450ad0b4c88b4950959fd49708969f02
I20260811 19:02:47.354483 29244 tablet_service.cc:1514] Processing DeleteTablet for tablet 450ad0b4c88b4950959fd49708969f02 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:02:47 UTC) from {username='kudu', principal='kudu/127.13.109.62@KRBTEST.COM'} at 127.0.0.1:54892
I20260811 19:02:47.354632 29094 tablet_service.cc:1514] Processing DeleteTablet for tablet 450ad0b4c88b4950959fd49708969f02 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:02:47 UTC) from {username='kudu', principal='kudu/127.13.109.62@KRBTEST.COM'} at 127.0.0.1:50900
I20260811 19:02:47.354817 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 28866
I20260811 19:02:47.356766 29362 tablet_replica.cc:330] T 450ad0b4c88b4950959fd49708969f02 P 7f911943ce2449aba06ac11215b59788: stopping tablet replica
I20260811 19:02:47.357201 29362 raft_consensus.cc:2229] T 450ad0b4c88b4950959fd49708969f02 P 7f911943ce2449aba06ac11215b59788 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:02:47.357599 29362 raft_consensus.cc:2258] T 450ad0b4c88b4950959fd49708969f02 P 7f911943ce2449aba06ac11215b59788 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:02:47.358461 29363 tablet_replica.cc:330] T 450ad0b4c88b4950959fd49708969f02 P fddff96ae4b84cdc86942277531be082: stopping tablet replica
I20260811 19:02:47.358814 29363 raft_consensus.cc:2229] T 450ad0b4c88b4950959fd49708969f02 P fddff96ae4b84cdc86942277531be082 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:02:47.359148 29363 raft_consensus.cc:2258] T 450ad0b4c88b4950959fd49708969f02 P fddff96ae4b84cdc86942277531be082 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:02:47.359748 29362 ts_tablet_manager.cc:1905] T 450ad0b4c88b4950959fd49708969f02 P 7f911943ce2449aba06ac11215b59788: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:02:47.360579 29363 ts_tablet_manager.cc:1905] T 450ad0b4c88b4950959fd49708969f02 P fddff96ae4b84cdc86942277531be082: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:02:47.363212 29362 ts_tablet_manager.cc:1918] T 450ad0b4c88b4950959fd49708969f02 P 7f911943ce2449aba06ac11215b59788: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:02:47.363351 29362 log.cc:1196] T 450ad0b4c88b4950959fd49708969f02 P 7f911943ce2449aba06ac11215b59788: Deleting WAL directory at /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-2/wal/wals/450ad0b4c88b4950959fd49708969f02
I20260811 19:02:47.363428 29363 ts_tablet_manager.cc:1918] T 450ad0b4c88b4950959fd49708969f02 P fddff96ae4b84cdc86942277531be082: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:02:47.363535 29363 log.cc:1196] T 450ad0b4c88b4950959fd49708969f02 P fddff96ae4b84cdc86942277531be082: Deleting WAL directory at /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-1/wal/wals/450ad0b4c88b4950959fd49708969f02
I20260811 19:02:47.363770 29362 ts_tablet_manager.cc:1939] T 450ad0b4c88b4950959fd49708969f02 P 7f911943ce2449aba06ac11215b59788: Deleting consensus metadata
I20260811 19:02:47.363858 29363 ts_tablet_manager.cc:1939] T 450ad0b4c88b4950959fd49708969f02 P fddff96ae4b84cdc86942277531be082: Deleting consensus metadata
I20260811 19:02:47.365244 28779 catalog_manager.cc:4841] TS 7f911943ce2449aba06ac11215b59788 (127.13.109.3:43983): tablet 450ad0b4c88b4950959fd49708969f02 (table test-table [id=af518d31ad664bd6bc5d2db566af3237]) successfully deleted
I20260811 19:02:47.365303 28778 catalog_manager.cc:4841] TS fddff96ae4b84cdc86942277531be082 (127.13.109.2:34823): tablet 450ad0b4c88b4950959fd49708969f02 (table test-table [id=af518d31ad664bd6bc5d2db566af3237]) successfully deleted
W20260811 19:02:47.365679 28778 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.109.1:42453 (error 108)
I20260811 19:02:47.365957 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 29024
I20260811 19:02:47.371603 29331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:47.371285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.1:42453 (local address 127.0.0.1:40008)
0811 19:02:47.371442 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:47.371518 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.1:42453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260811 19:02:47.371941 28778 catalog_manager.cc:4584] TS a088e3d057b849d593f4d1972111e6c8 (127.13.109.1:42453): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 450ad0b4c88b4950959fd49708969f02: Network error: Client connection negotiation failed: client connection to 127.13.109.1:42453: connect: Connection refused (error 111)
I20260811 19:02:47.377099 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 29177
I20260811 19:02:47.385825 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 28761
I20260811 19:02:47.395520 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.62:37073
--webserver_interface=127.13.109.62
--webserver_port=39633
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab
--principal=kudu/127.13.109.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:38669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.109.62:37073
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:02:47.517659 29365 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:02:47.518110 29365 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:02:47.518201 29365 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:02:47.522540 29365 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:02:47.522663 29365 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:02:47.522701 29365 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:02:47.522745 29365 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:02:47.522778 29365 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:02:47.528786 29365 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:38669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.13.109.62:37073
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab
--principal=kudu/127.13.109.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.62:37073
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.109.62
--webserver_port=39633
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29365
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:02:47.530134 29365 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:02:47.531267 29365 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:02:47.547307 29371 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:02:47.547353 29372 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:02:47.547477 29365 server_base.cc:1034] running on GCE node
W20260811 19:02:47.547705 29374 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:02:47.548411 29365 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:02:47.549714 29365 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:02:47.550940 29365 hybrid_clock.cc:648] HybridClock initialized: now 1786474967550904 us; error 45 us; skew 500 ppm
Aug 11 19:02:47 dist-test-slave-0kls krb5kdc[28748](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474967, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:02:47.554147 29365 init.cc:377] Logged in from keytab as kudu/127.13.109.62@KRBTEST.COM (short username kudu)
I20260811 19:02:47.555248 29365 webserver.cc:460] Webserver started at http://127.13.109.62:39633/ using document root <none> and password file <none>
I20260811 19:02:47.555899 29365 fs_manager.cc:359] Metadata directory not provided
I20260811 19:02:47.556022 29365 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:02:47.560449 29365 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 19:02:47.562986 29382 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:47.564502 29365 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 19:02:47.564687 29365 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/master-0/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/master-0/wal
uuid: "df03a25279794bc29ae9e20f2d28c530"
format_stamp: "Formatted at 2026-08-11 19:02:46 on dist-test-slave-0kls"
server_key: "f63b3ab253bfa165821434b618940ab1"
server_key_iv: "b4834185cf87588d47aea14f01621ed8"
server_key_version: "encryptionkey@0"
I20260811 19:02:47.565382 29365 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:02:47.583071 29365 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:02:47.586617 29365 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:02:47.586858 29365 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:02:47.594633 29365 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.62:37073
I20260811 19:02:47.594929 29438 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.62:37073 every 8 connection(s)
I20260811 19:02:47.595913 29365 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
I20260811 19:02:47.600762 29439 sys_catalog.cc:263] Verifying existing consensus state
I20260811 19:02:47.602639 29439 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df03a25279794bc29ae9e20f2d28c530: Bootstrap starting.
I20260811 19:02:47.604189 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 29365
I20260811 19:02:47.604672 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.1:42453
--local_ip_for_outbound_sockets=127.13.109.1
--tserver_master_addrs=127.13.109.62:37073
--webserver_port=46621
--webserver_interface=127.13.109.1
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.1.keytab
--principal=kudu/127.13.109.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:38669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 19:02:47.621543 29439 log.cc:824] T 00000000000000000000000000000000 P df03a25279794bc29ae9e20f2d28c530: Log is configured to *not* fsync() on all Append() calls
I20260811 19:02:47.631090 29439 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df03a25279794bc29ae9e20f2d28c530: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 19:02:47.631527 29439 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df03a25279794bc29ae9e20f2d28c530: Bootstrap complete.
I20260811 19:02:47.635986 29439 raft_consensus.cc:348] T 00000000000000000000000000000000 P df03a25279794bc29ae9e20f2d28c530 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df03a25279794bc29ae9e20f2d28c530" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 37073 } }
I20260811 19:02:47.636458 29439 raft_consensus.cc:729] T 00000000000000000000000000000000 P df03a25279794bc29ae9e20f2d28c530 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: df03a25279794bc29ae9e20f2d28c530, State: Initialized, Role: FOLLOWER
I20260811 19:02:47.637054 29439 consensus_queue.cc:260] T 00000000000000000000000000000000 P df03a25279794bc29ae9e20f2d28c530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df03a25279794bc29ae9e20f2d28c530" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 37073 } }
I20260811 19:02:47.637233 29439 raft_consensus.cc:388] T 00000000000000000000000000000000 P df03a25279794bc29ae9e20f2d28c530 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:02:47.637323 29439 raft_consensus.cc:482] T 00000000000000000000000000000000 P df03a25279794bc29ae9e20f2d28c530 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:02:47.637455 29439 raft_consensus.cc:3037] T 00000000000000000000000000000000 P df03a25279794bc29ae9e20f2d28c530 [term 1 FOLLOWER]: Advancing to term 2
I20260811 19:02:47.639667 29439 raft_consensus.cc:504] T 00000000000000000000000000000000 P df03a25279794bc29ae9e20f2d28c530 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df03a25279794bc29ae9e20f2d28c530" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 37073 } }
I20260811 19:02:47.640070 29439 leader_election.cc:302] T 00000000000000000000000000000000 P df03a25279794bc29ae9e20f2d28c530 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: df03a25279794bc29ae9e20f2d28c530; no voters:
I20260811 19:02:47.640432 29439 leader_election.cc:288] T 00000000000000000000000000000000 P df03a25279794bc29ae9e20f2d28c530 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 19:02:47.640607 29443 raft_consensus.cc:2781] T 00000000000000000000000000000000 P df03a25279794bc29ae9e20f2d28c530 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 19:02:47.640893 29443 raft_consensus.cc:686] T 00000000000000000000000000000000 P df03a25279794bc29ae9e20f2d28c530 [term 2 LEADER]: Becoming Leader. State: Replica: df03a25279794bc29ae9e20f2d28c530, State: Running, Role: LEADER
I20260811 19:02:47.641243 29443 consensus_queue.cc:237] T 00000000000000000000000000000000 P df03a25279794bc29ae9e20f2d28c530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df03a25279794bc29ae9e20f2d28c530" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 37073 } }
I20260811 19:02:47.641928 29439 sys_catalog.cc:564] T 00000000000000000000000000000000 P df03a25279794bc29ae9e20f2d28c530 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:02:47.642957 29445 sys_catalog.cc:455] T 00000000000000000000000000000000 P df03a25279794bc29ae9e20f2d28c530 [sys.catalog]: SysCatalogTable state changed. Reason: New leader df03a25279794bc29ae9e20f2d28c530. Latest consensus state: current_term: 2 leader_uuid: "df03a25279794bc29ae9e20f2d28c530" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df03a25279794bc29ae9e20f2d28c530" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 37073 } } }
I20260811 19:02:47.643021 29444 sys_catalog.cc:455] T 00000000000000000000000000000000 P df03a25279794bc29ae9e20f2d28c530 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "df03a25279794bc29ae9e20f2d28c530" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df03a25279794bc29ae9e20f2d28c530" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 37073 } } }
I20260811 19:02:47.643110 29445 sys_catalog.cc:458] T 00000000000000000000000000000000 P df03a25279794bc29ae9e20f2d28c530 [sys.catalog]: This master's current role is: LEADER
I20260811 19:02:47.643138 29444 sys_catalog.cc:458] T 00000000000000000000000000000000 P df03a25279794bc29ae9e20f2d28c530 [sys.catalog]: This master's current role is: LEADER
I20260811 19:02:47.643496 29454 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:02:47.648828 29454 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:02:47.649621 29454 catalog_manager.cc:1259] Loaded cluster ID: 77858b6acb72403bb610ad5d3895ea9a
I20260811 19:02:47.649695 29454 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:02:47.651392 29454 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260811 19:02:47.655936 29454 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260811 19:02:47.662470 29454 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:02:47.663506 29454 catalog_manager.cc:5879] T 00000000000000000000000000000000 P df03a25279794bc29ae9e20f2d28c530: Loaded TSK: 0
I20260811 19:02:47.665352 29454 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 19:02:47.735445 29441 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:02:47.735769 29441 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:02:47.735852 29441 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:02:47.740229 29441 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:02:47.740342 29441 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:02:47.740444 29441 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.1
I20260811 19:02:47.746706 29441 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:38669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.1.keytab
--principal=kudu/127.13.109.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.1:42453
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.109.1
--webserver_port=46621
--tserver_master_addrs=127.13.109.62:37073
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29441
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.1
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:02:47.747921 29441 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:02:47.748992 29441 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:02:47.759380 29473 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:02:47.759384 29475 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:02:47.759380 29472 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:02:47.759949 29441 server_base.cc:1034] running on GCE node
I20260811 19:02:47.760434 29441 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:02:47.761217 29441 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:02:47.762400 29441 hybrid_clock.cc:648] HybridClock initialized: now 1786474967762370 us; error 52 us; skew 500 ppm
Aug 11 19:02:47 dist-test-slave-0kls krb5kdc[28748](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474967, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:02:47.765461 29441 init.cc:377] Logged in from keytab as kudu/127.13.109.1@KRBTEST.COM (short username kudu)
I20260811 19:02:47.766597 29441 webserver.cc:460] Webserver started at http://127.13.109.1:46621/ using document root <none> and password file <none>
I20260811 19:02:47.767206 29441 fs_manager.cc:359] Metadata directory not provided
I20260811 19:02:47.767329 29441 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:02:47.771888 29441 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 19:02:47.774305 29482 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:47.775731 29441 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 19:02:47.775846 29441 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-0/wal
uuid: "a088e3d057b849d593f4d1972111e6c8"
format_stamp: "Formatted at 2026-08-11 19:02:46 on dist-test-slave-0kls"
server_key: "ef85a7a5af195b603801d7ede48c299e"
server_key_iv: "2dd632a0f7272b236484ff4c805f4ce1"
server_key_version: "encryptionkey@0"
I20260811 19:02:47.776470 29441 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:02:47.791256 29441 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:02:47.794641 29441 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:02:47.794860 29441 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:02:47.795543 29441 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:02:47.796731 29490 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 19:02:47.801568 29441 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 19:02:47.801685 29441 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260811 19:02:47.801836 29441 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 19:02:47.803747 29441 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 19:02:47.803828 29441 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260811 19:02:47.804137 29490 tablet_bootstrap.cc:492] T 450ad0b4c88b4950959fd49708969f02 P a088e3d057b849d593f4d1972111e6c8: Bootstrap starting.
I20260811 19:02:47.814929 29441 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.1:42453
I20260811 19:02:47.815016 29605 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.1:42453 every 8 connection(s)
I20260811 19:02:47.815975 29441 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
I20260811 19:02:47.823781 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 29441
I20260811 19:02:47.824047 29490 log.cc:824] T 450ad0b4c88b4950959fd49708969f02 P a088e3d057b849d593f4d1972111e6c8: Log is configured to *not* fsync() on all Append() calls
I20260811 19:02:47.824339 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.2:34823
--local_ip_for_outbound_sockets=127.13.109.2
--tserver_master_addrs=127.13.109.62:37073
--webserver_port=38343
--webserver_interface=127.13.109.2
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.2.keytab
--principal=kudu/127.13.109.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:38669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 19:02:47 dist-test-slave-0kls krb5kdc[28748](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474967, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.1@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
I20260811 19:02:47.834120 29490 tablet_bootstrap.cc:492] T 450ad0b4c88b4950959fd49708969f02 P a088e3d057b849d593f4d1972111e6c8: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 19:02:47.834689 29490 tablet_bootstrap.cc:492] T 450ad0b4c88b4950959fd49708969f02 P a088e3d057b849d593f4d1972111e6c8: Bootstrap complete.
I20260811 19:02:47.835542 29609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:47.818134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:37073 (local address 127.13.109.1:32925)
0811 19:02:47.818800 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:47.818857 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 19:02:47.819709 (+ 852us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:02:47.821281 (+ 1572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:02:47.821288 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:02:47.821684 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:02:47.822388 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:47.822406 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:47.824328 (+ 1922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:47.824332 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:47.826395 (+ 2063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:47.826408 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:47.827501 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:47.827508 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:47.827746 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:47.828490 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:02:47.828508 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:02:47.830080 (+ 1572us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:02:47.832294 (+ 2214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:02:47.832300 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:02:47.832314 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:02:47.832587 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:02:47.832972 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:02:47.832976 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:02:47.832978 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:02:47.833130 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:02:47.833745 (+ 615us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:02:47.833752 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:02:47.834035 (+ 283us) client_negotiation.cc:769] Sending connection context
0811 19:02:47.834347 (+ 312us) client_negotiation.cc:240] Negotiation successful
0811 19:02:47.834657 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":169,"threads_started":1}
I20260811 19:02:47.836323 29490 ts_tablet_manager.cc:1397] T 450ad0b4c88b4950959fd49708969f02 P a088e3d057b849d593f4d1972111e6c8: Time spent bootstrapping tablet: real 0.032s user 0.026s sys 0.004s
I20260811 19:02:47.837158 29608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:47.818186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.1:32925 (local address 127.13.109.62:37073)
0811 19:02:47.818658 (+ 472us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:47.818683 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:47.819001 (+ 318us) server_negotiation.cc:407] Connection header received
0811 19:02:47.820274 (+ 1273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:47.820300 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:47.820783 (+ 483us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:47.821216 (+ 433us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:02:47.822599 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:47.824128 (+ 1529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:47.826566 (+ 2438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:47.827319 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:47.827501 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:47.830319 (+ 2818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:02:47.830324 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:02:47.830337 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:02:47.830369 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:02:47.832153 (+ 1784us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:47.832731 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:47.832737 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:47.832741 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:47.832865 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:47.833247 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:47.833250 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:47.833251 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:47.833607 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:02:47.833773 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:47.836185 (+ 2412us) server_negotiation.cc:299] Negotiation successful
0811 19:02:47.836508 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":164,"threads_started":1}
I20260811 19:02:47.839602 29606 heartbeater.cc:347] Connected to a master server at 127.13.109.62:37073
I20260811 19:02:47.839869 29606 heartbeater.cc:464] Registering TS with master...
I20260811 19:02:47.840400 29606 heartbeater.cc:511] Master 127.13.109.62:37073 requested a full tablet report, sending...
I20260811 19:02:47.840282 29490 raft_consensus.cc:348] T 450ad0b4c88b4950959fd49708969f02 P a088e3d057b849d593f4d1972111e6c8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f911943ce2449aba06ac11215b59788" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43983 } } peers { permanent_uuid: "fddff96ae4b84cdc86942277531be082" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34823 } } peers { permanent_uuid: "a088e3d057b849d593f4d1972111e6c8" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 42453 } }
I20260811 19:02:47.840821 29490 raft_consensus.cc:729] T 450ad0b4c88b4950959fd49708969f02 P a088e3d057b849d593f4d1972111e6c8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a088e3d057b849d593f4d1972111e6c8, State: Initialized, Role: FOLLOWER
I20260811 19:02:47.841475 29490 consensus_queue.cc:260] T 450ad0b4c88b4950959fd49708969f02 P a088e3d057b849d593f4d1972111e6c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f911943ce2449aba06ac11215b59788" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43983 } } peers { permanent_uuid: "fddff96ae4b84cdc86942277531be082" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34823 } } peers { permanent_uuid: "a088e3d057b849d593f4d1972111e6c8" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 42453 } }
I20260811 19:02:47.842661 29490 ts_tablet_manager.cc:1428] T 450ad0b4c88b4950959fd49708969f02 P a088e3d057b849d593f4d1972111e6c8: Time spent starting tablet: real 0.006s user 0.004s sys 0.004s
I20260811 19:02:47.844583 29401 ts_manager.cc:194] Registered new tserver with Master: a088e3d057b849d593f4d1972111e6c8 (127.13.109.1:42453)
I20260811 19:02:47.848407 29401 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.109.1@KRBTEST.COM'} at 127.13.109.1:32925
I20260811 19:02:47.851300 29606 heartbeater.cc:503] Master 127.13.109.62:37073 was elected leader, sending a full tablet report...
Aug 11 19:02:47 dist-test-slave-0kls krb5kdc[28748](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474967, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.62@KRBTEST.COM for kudu/127.13.109.1@KRBTEST.COM
I20260811 19:02:47.857900 29613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:47.847407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.1:42453 (local address 127.0.0.1:40018)
0811 19:02:47.847857 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:47.847915 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 19:02:47.848044 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:02:47.848681 (+ 637us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:02:47.848687 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:02:47.849126 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:02:47.849269 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:47.849280 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:47.850356 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:47.850359 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:47.851084 (+ 725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:47.851095 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:47.851799 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:47.851802 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:47.851923 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:47.852569 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:02:47.852584 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:02:47.854157 (+ 1573us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:02:47.856001 (+ 1844us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:02:47.856011 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:02:47.856017 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:02:47.856266 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:02:47.856585 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:02:47.856589 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:02:47.856591 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:02:47.856684 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:02:47.857172 (+ 488us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:02:47.857178 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:02:47.857307 (+ 129us) client_negotiation.cc:769] Sending connection context
0811 19:02:47.857447 (+ 140us) client_negotiation.cc:240] Negotiation successful
0811 19:02:47.857663 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":117,"threads_started":1}
I20260811 19:02:47.857919 29614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:47.847496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40018 (local address 127.13.109.1:42453)
0811 19:02:47.848171 (+ 675us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:47.848181 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:47.848203 (+ 22us) server_negotiation.cc:407] Connection header received
0811 19:02:47.848267 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:47.848272 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:47.848472 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:47.848658 (+ 186us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:02:47.849402 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:47.850235 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:47.851207 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:47.851641 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:47.851782 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:47.854305 (+ 2523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:02:47.854322 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:02:47.854333 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:02:47.854356 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:02:47.855871 (+ 1515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:47.856389 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:47.856395 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:47.856397 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:47.856485 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:47.856777 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:47.856782 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:47.856783 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:47.857033 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:02:47.857175 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:47.857544 (+ 369us) server_negotiation.cc:299] Negotiation successful
0811 19:02:47.857723 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":555,"thread_start_us":105,"threads_started":1}
I20260811 19:02:47.858920 29535 tablet_service.cc:1514] Processing DeleteTablet for tablet 450ad0b4c88b4950959fd49708969f02 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:02:47 UTC) from {username='kudu', principal='kudu/127.13.109.62@KRBTEST.COM'} at 127.0.0.1:40018
I20260811 19:02:47.859800 29615 tablet_replica.cc:330] stopping tablet replica
I20260811 19:02:47.860111 29615 raft_consensus.cc:2229] T 450ad0b4c88b4950959fd49708969f02 P a088e3d057b849d593f4d1972111e6c8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:02:47.860375 29615 raft_consensus.cc:2258] T 450ad0b4c88b4950959fd49708969f02 P a088e3d057b849d593f4d1972111e6c8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:02:47.861435 29615 ts_tablet_manager.cc:1905] T 450ad0b4c88b4950959fd49708969f02 P a088e3d057b849d593f4d1972111e6c8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:02:47.865026 29615 ts_tablet_manager.cc:1918] T 450ad0b4c88b4950959fd49708969f02 P a088e3d057b849d593f4d1972111e6c8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:02:47.865115 29615 log.cc:1196] T 450ad0b4c88b4950959fd49708969f02 P a088e3d057b849d593f4d1972111e6c8: Deleting WAL directory at /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-0/wal/wals/450ad0b4c88b4950959fd49708969f02
I20260811 19:02:47.865392 29615 ts_tablet_manager.cc:1939] T 450ad0b4c88b4950959fd49708969f02 P a088e3d057b849d593f4d1972111e6c8: Deleting consensus metadata
I20260811 19:02:47.867219 29383 catalog_manager.cc:4841] TS a088e3d057b849d593f4d1972111e6c8 (127.13.109.1:42453): tablet 450ad0b4c88b4950959fd49708969f02 (table test-table [id=af518d31ad664bd6bc5d2db566af3237]) successfully deleted
W20260811 19:02:47.955475 29611 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:02:47.955823 29611 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:02:47.955906 29611 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:02:47.960165 29611 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:02:47.960290 29611 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:02:47.960395 29611 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.2
I20260811 19:02:47.966202 29611 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:38669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.2.keytab
--principal=kudu/127.13.109.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.2:34823
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.109.2
--webserver_port=38343
--tserver_master_addrs=127.13.109.62:37073
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29611
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.2
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:02:47.967519 29611 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:02:47.968683 29611 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:02:47.979516 29622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:02:47.979533 29620 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:02:47.979533 29624 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:02:47.979594 29611 server_base.cc:1034] running on GCE node
I20260811 19:02:47.980341 29611 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:02:47.981123 29611 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:02:47.982321 29611 hybrid_clock.cc:648] HybridClock initialized: now 1786474967982304 us; error 28 us; skew 500 ppm
Aug 11 19:02:47 dist-test-slave-0kls krb5kdc[28748](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474967, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:02:47.985316 29611 init.cc:377] Logged in from keytab as kudu/127.13.109.2@KRBTEST.COM (short username kudu)
I20260811 19:02:47.986382 29611 webserver.cc:460] Webserver started at http://127.13.109.2:38343/ using document root <none> and password file <none>
I20260811 19:02:47.987022 29611 fs_manager.cc:359] Metadata directory not provided
I20260811 19:02:47.987119 29611 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:02:47.991484 29611 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 19:02:47.993919 29631 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:47.995296 29611 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 19:02:47.995411 29611 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-1/wal
uuid: "fddff96ae4b84cdc86942277531be082"
format_stamp: "Formatted at 2026-08-11 19:02:46 on dist-test-slave-0kls"
server_key: "75888febf87c85be2459138e377876ca"
server_key_iv: "d05678c0d1c93a01a233a1475e19d331"
server_key_version: "encryptionkey@0"
I20260811 19:02:47.995999 29611 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:02:48.014504 29611 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:02:48.017951 29611 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:02:48.018152 29611 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:02:48.018780 29611 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:02:48.019699 29611 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:02:48.019773 29611 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:48.019820 29611 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:02:48.019841 29611 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:48.030356 29611 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.2:34823
I20260811 19:02:48.030376 29758 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.2:34823 every 8 connection(s)
I20260811 19:02:48.031522 29611 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
I20260811 19:02:48.034868 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 29611
I20260811 19:02:48.035405 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.3:43983
--local_ip_for_outbound_sockets=127.13.109.3
--tserver_master_addrs=127.13.109.62:37073
--webserver_port=39733
--webserver_interface=127.13.109.3
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.3.keytab
--principal=kudu/127.13.109.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:38669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 19:02:48 dist-test-slave-0kls krb5kdc[28748](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474967, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.2@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
I20260811 19:02:48.048955 29608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:48.033552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.2:46123 (local address 127.13.109.62:37073)
0811 19:02:48.033721 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:48.033725 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:48.035380 (+ 1655us) server_negotiation.cc:407] Connection header received
0811 19:02:48.036990 (+ 1610us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:48.036997 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:48.037080 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:48.037199 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:02:48.039224 (+ 2025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:48.040106 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:48.041439 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:48.041704 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:48.041862 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:48.044922 (+ 3060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:02:48.044925 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:02:48.044927 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:02:48.044957 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:02:48.046589 (+ 1632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:48.047207 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:48.047213 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:48.047219 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:48.047306 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:48.047711 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:48.047714 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:48.047716 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:48.047927 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:02:48.048039 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:48.048620 (+ 581us) server_negotiation.cc:299] Negotiation successful
0811 19:02:48.048779 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 19:02:48.049955 29761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:48.034097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:37073 (local address 127.13.109.2:46123)
0811 19:02:48.035138 (+ 1041us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:48.035220 (+ 82us) client_negotiation.cc:174] Beginning negotiation
0811 19:02:48.036667 (+ 1447us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:02:48.037398 (+ 731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:02:48.037405 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:02:48.038118 (+ 713us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:02:48.038962 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:48.038990 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:48.040293 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:48.040295 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:48.041266 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:48.041278 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:48.041946 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:48.041951 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:48.042148 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:48.042909 (+ 761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:02:48.042928 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:02:48.044693 (+ 1765us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:02:48.046748 (+ 2055us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:02:48.046755 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:02:48.046778 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:02:48.047083 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:02:48.047411 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:02:48.047415 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:02:48.047417 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:02:48.047585 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:02:48.048057 (+ 472us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:02:48.048065 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:02:48.048361 (+ 296us) client_negotiation.cc:769] Sending connection context
0811 19:02:48.048576 (+ 215us) client_negotiation.cc:240] Negotiation successful
0811 19:02:48.048962 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":166,"threads_started":1}
I20260811 19:02:48.051533 29759 heartbeater.cc:347] Connected to a master server at 127.13.109.62:37073
I20260811 19:02:48.051851 29759 heartbeater.cc:464] Registering TS with master...
I20260811 19:02:48.052455 29759 heartbeater.cc:511] Master 127.13.109.62:37073 requested a full tablet report, sending...
I20260811 19:02:48.053812 29401 ts_manager.cc:194] Registered new tserver with Master: fddff96ae4b84cdc86942277531be082 (127.13.109.2:34823)
I20260811 19:02:48.054648 29401 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.109.2@KRBTEST.COM'} at 127.13.109.2:46123
W20260811 19:02:48.164965 29762 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:02:48.165280 29762 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:02:48.165326 29762 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:02:48.169870 29762 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:02:48.169956 29762 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:02:48.170024 29762 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.3
I20260811 19:02:48.176836 29762 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:38669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.3.keytab
--principal=kudu/127.13.109.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.3:43983
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.109.3
--webserver_port=39733
--tserver_master_addrs=127.13.109.62:37073
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29762
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.3
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:02:48.178077 29762 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:02:48.179167 29762 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:02:48.189968 29767 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:02:48.189970 29769 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:02:48.190032 29762 server_base.cc:1034] running on GCE node
W20260811 19:02:48.190323 29771 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:02:48.190758 29762 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:02:48.191474 29762 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:02:48.192698 29762 hybrid_clock.cc:648] HybridClock initialized: now 1786474968192651 us; error 52 us; skew 500 ppm
Aug 11 19:02:48 dist-test-slave-0kls krb5kdc[28748](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474968, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:02:48.195726 29762 init.cc:377] Logged in from keytab as kudu/127.13.109.3@KRBTEST.COM (short username kudu)
I20260811 19:02:48.196661 29762 webserver.cc:460] Webserver started at http://127.13.109.3:39733/ using document root <none> and password file <none>
I20260811 19:02:48.197237 29762 fs_manager.cc:359] Metadata directory not provided
I20260811 19:02:48.197314 29762 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:02:48.201678 29762 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260811 19:02:48.204001 29779 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:48.205397 29762 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 19:02:48.205637 29762 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-2/wal
uuid: "7f911943ce2449aba06ac11215b59788"
format_stamp: "Formatted at 2026-08-11 19:02:47 on dist-test-slave-0kls"
server_key: "b58775d7cbbb98b1c1833ff3b12d2d18"
server_key_iv: "72f02612bbefbb73059d05dc5554a9f8"
server_key_version: "encryptionkey@0"
I20260811 19:02:48.206326 29762 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:02:48.224187 29762 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:02:48.227867 29762 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:02:48.228091 29762 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:02:48.228765 29762 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:02:48.229686 29762 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:02:48.229737 29762 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:48.229837 29762 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:02:48.229882 29762 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:48.239647 29762 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.3:43983
I20260811 19:02:48.239674 29903 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.3:43983 every 8 connection(s)
I20260811 19:02:48.240849 29762 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
I20260811 19:02:48.245482 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 29762
Aug 11 19:02:48 dist-test-slave-0kls krb5kdc[28748](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474968, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.3@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
I20260811 19:02:48.254916 29907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:48.246246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36136 (local address 127.13.109.62:37073)
0811 19:02:48.246482 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:48.246487 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:48.246670 (+ 183us) server_negotiation.cc:407] Connection header received
0811 19:02:48.246823 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:48.246827 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:48.246875 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:48.246975 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:02:48.247748 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:48.248259 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:48.249117 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:48.249345 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:48.249458 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:48.251045 (+ 1587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:02:48.251048 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:02:48.251050 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:02:48.251077 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:02:48.252854 (+ 1777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:48.253413 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:48.253416 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:48.253417 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:48.253500 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:48.253870 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:48.253873 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:48.253874 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:48.254075 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:02:48.254378 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:48.254583 (+ 205us) server_negotiation.cc:299] Negotiation successful
0811 19:02:48.254722 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":78,"threads_started":1}
I20260811 19:02:48.256309 29608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:48.242900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.3:37251 (local address 127.13.109.62:37073)
0811 19:02:48.243041 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:48.243051 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:48.244110 (+ 1059us) server_negotiation.cc:407] Connection header received
0811 19:02:48.245563 (+ 1453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:48.245567 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:48.245634 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:48.245743 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:02:48.247209 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:48.247731 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:48.248913 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:48.249136 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:48.249252 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:48.252131 (+ 2879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:02:48.252135 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:02:48.252137 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:02:48.252177 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:02:48.253902 (+ 1725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:48.254481 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:48.254485 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:48.254487 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:48.254575 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:48.255017 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:48.255022 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:48.255024 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:48.255222 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:02:48.255336 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:48.256051 (+ 715us) server_negotiation.cc:299] Negotiation successful
0811 19:02:48.256183 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 19:02:48.257622 29906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:48.243363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:37073 (local address 127.13.109.3:37251)
0811 19:02:48.243896 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:48.243999 (+ 103us) client_negotiation.cc:174] Beginning negotiation
0811 19:02:48.245342 (+ 1343us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:02:48.245971 (+ 629us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:02:48.245982 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:02:48.246515 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:02:48.247047 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:48.247062 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:48.247861 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:48.247865 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:48.248771 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:48.248782 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:48.249305 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:48.249309 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:48.249493 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:48.250305 (+ 812us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:02:48.250324 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:02:48.251928 (+ 1604us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:02:48.254022 (+ 2094us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:02:48.254030 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:02:48.254052 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:02:48.254365 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:02:48.254676 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:02:48.254679 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:02:48.254681 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:02:48.254905 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:02:48.255359 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:02:48.255365 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:02:48.255741 (+ 376us) client_negotiation.cc:769] Sending connection context
0811 19:02:48.255978 (+ 237us) client_negotiation.cc:240] Negotiation successful
0811 19:02:48.256371 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":122,"threads_started":1}
I20260811 19:02:48.258970 29904 heartbeater.cc:347] Connected to a master server at 127.13.109.62:37073
I20260811 19:02:48.259264 29904 heartbeater.cc:464] Registering TS with master...
I20260811 19:02:48.259845 29904 heartbeater.cc:511] Master 127.13.109.62:37073 requested a full tablet report, sending...
I20260811 19:02:48.261022 29401 ts_manager.cc:194] Registered new tserver with Master: 7f911943ce2449aba06ac11215b59788 (127.13.109.3:43983)
I20260811 19:02:48.261649 29401 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.109.3@KRBTEST.COM'} at 127.13.109.3:37251
I20260811 19:02:48.269289 13748 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 19:02:48.273886 29608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:48.270084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36142 (local address 127.13.109.62:37073)
0811 19:02:48.270231 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:48.270234 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:48.270381 (+ 147us) server_negotiation.cc:407] Connection header received
0811 19:02:48.270533 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:48.270536 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:48.270657 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:48.270757 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:02:48.270994 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:48.271529 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:48.272404 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:48.272623 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:48.272709 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:48.273009 (+ 300us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:02:48.273471 (+ 462us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:02:48.273566 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:48.273626 (+ 60us) server_negotiation.cc:299] Negotiation successful
0811 19:02:48.273679 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 19:02:48.274670 29401 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:36142:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 19:02:48.278256 29401 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 19:02:48.288363 29535 tablet_service.cc:1467] Processing CreateTablet for tablet 9768ba0ee6164761b54ed6fdcd7865b2 (DEFAULT_TABLE table=test-table [id=bfed1122de8341f1bcff80f7e70a6af1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:02:48.289064 29535 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9768ba0ee6164761b54ed6fdcd7865b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:02:48.291800 29490 tablet_bootstrap.cc:492] T 9768ba0ee6164761b54ed6fdcd7865b2 P a088e3d057b849d593f4d1972111e6c8: Bootstrap starting.
I20260811 19:02:48.292776 29490 tablet_bootstrap.cc:654] T 9768ba0ee6164761b54ed6fdcd7865b2 P a088e3d057b849d593f4d1972111e6c8: Neither blocks nor log segments found. Creating new log.
I20260811 19:02:48.293243 29613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:48.286427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.2:34823 (local address 127.0.0.1:50912)
0811 19:02:48.286541 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:48.286557 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 19:02:48.286642 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:02:48.287897 (+ 1255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:02:48.287901 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:02:48.287920 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:02:48.288040 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:48.288046 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:48.289566 (+ 1520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:48.289571 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:48.291198 (+ 1627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:48.291211 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:48.292714 (+ 1503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:48.292718 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:48.292819 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:48.292894 (+ 75us) client_negotiation.cc:769] Sending connection context
0811 19:02:48.292981 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 19:02:48.293097 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32,"mutex_wait_us":178}
I20260811 19:02:48.293258 29916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:48.286791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50912 (local address 127.13.109.2:34823)
0811 19:02:48.287190 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:48.287200 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:48.287221 (+ 21us) server_negotiation.cc:407] Connection header received
0811 19:02:48.287288 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:48.287293 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:48.287534 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:48.287736 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:02:48.288188 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:48.289440 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:48.291318 (+ 1878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:48.292560 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:48.292711 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:48.292902 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:48.293015 (+ 113us) server_negotiation.cc:299] Negotiation successful
0811 19:02:48.293113 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":169,"threads_started":1}
I20260811 19:02:48.293408 29917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:48.287220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54908 (local address 127.13.109.3:43983)
0811 19:02:48.287549 (+ 329us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:48.287561 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:48.287585 (+ 24us) server_negotiation.cc:407] Connection header received
0811 19:02:48.287666 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:48.287673 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:48.287886 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:48.288046 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:02:48.288384 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:48.289584 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:48.291468 (+ 1884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:48.292323 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:48.292434 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:48.292602 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:48.293120 (+ 518us) server_negotiation.cc:299] Negotiation successful
0811 19:02:48.293243 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":223,"thread_start_us":141,"threads_started":1}
I20260811 19:02:48.293901 29915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:48.286662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.3:43983 (local address 127.0.0.1:54908)
0811 19:02:48.286899 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:48.286909 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 19:02:48.286965 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:02:48.288027 (+ 1062us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:02:48.288030 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:02:48.288043 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:02:48.288136 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:48.288142 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:48.289719 (+ 1577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:48.289722 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:48.291146 (+ 1424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:48.291165 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:48.292678 (+ 1513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:48.292685 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:48.292817 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:48.292911 (+ 94us) client_negotiation.cc:769] Sending connection context
0811 19:02:48.292997 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 19:02:48.293732 (+ 735us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"mutex_wait_us":705,"thread_start_us":122,"threads_started":1}
I20260811 19:02:48.295146 29490 tablet_bootstrap.cc:492] T 9768ba0ee6164761b54ed6fdcd7865b2 P a088e3d057b849d593f4d1972111e6c8: No bootstrap required, opened a new log
I20260811 19:02:48.295246 29490 ts_tablet_manager.cc:1397] T 9768ba0ee6164761b54ed6fdcd7865b2 P a088e3d057b849d593f4d1972111e6c8: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 19:02:48.295677 29490 raft_consensus.cc:348] T 9768ba0ee6164761b54ed6fdcd7865b2 P a088e3d057b849d593f4d1972111e6c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fddff96ae4b84cdc86942277531be082" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34823 } } peers { permanent_uuid: "7f911943ce2449aba06ac11215b59788" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43983 } } peers { permanent_uuid: "a088e3d057b849d593f4d1972111e6c8" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 42453 } }
I20260811 19:02:48.295782 29490 raft_consensus.cc:374] T 9768ba0ee6164761b54ed6fdcd7865b2 P a088e3d057b849d593f4d1972111e6c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:02:48.295810 29490 raft_consensus.cc:729] T 9768ba0ee6164761b54ed6fdcd7865b2 P a088e3d057b849d593f4d1972111e6c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a088e3d057b849d593f4d1972111e6c8, State: Initialized, Role: FOLLOWER
I20260811 19:02:48.295960 29490 consensus_queue.cc:260] T 9768ba0ee6164761b54ed6fdcd7865b2 P a088e3d057b849d593f4d1972111e6c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fddff96ae4b84cdc86942277531be082" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34823 } } peers { permanent_uuid: "7f911943ce2449aba06ac11215b59788" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43983 } } peers { permanent_uuid: "a088e3d057b849d593f4d1972111e6c8" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 42453 } }
I20260811 19:02:48.296257 29490 ts_tablet_manager.cc:1428] T 9768ba0ee6164761b54ed6fdcd7865b2 P a088e3d057b849d593f4d1972111e6c8: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 19:02:48.297233 29682 tablet_service.cc:1467] Processing CreateTablet for tablet 9768ba0ee6164761b54ed6fdcd7865b2 (DEFAULT_TABLE table=test-table [id=bfed1122de8341f1bcff80f7e70a6af1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:02:48.298497 29682 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9768ba0ee6164761b54ed6fdcd7865b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:02:48.298215 29835 tablet_service.cc:1467] Processing CreateTablet for tablet 9768ba0ee6164761b54ed6fdcd7865b2 (DEFAULT_TABLE table=test-table [id=bfed1122de8341f1bcff80f7e70a6af1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:02:48.299203 29835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9768ba0ee6164761b54ed6fdcd7865b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:02:48.306049 29921 tablet_bootstrap.cc:492] T 9768ba0ee6164761b54ed6fdcd7865b2 P 7f911943ce2449aba06ac11215b59788: Bootstrap starting.
I20260811 19:02:48.306049 29922 tablet_bootstrap.cc:492] T 9768ba0ee6164761b54ed6fdcd7865b2 P fddff96ae4b84cdc86942277531be082: Bootstrap starting.
I20260811 19:02:48.308658 29921 tablet_bootstrap.cc:654] T 9768ba0ee6164761b54ed6fdcd7865b2 P 7f911943ce2449aba06ac11215b59788: Neither blocks nor log segments found. Creating new log.
I20260811 19:02:48.308645 29922 tablet_bootstrap.cc:654] T 9768ba0ee6164761b54ed6fdcd7865b2 P fddff96ae4b84cdc86942277531be082: Neither blocks nor log segments found. Creating new log.
I20260811 19:02:48.309515 29921 log.cc:824] T 9768ba0ee6164761b54ed6fdcd7865b2 P 7f911943ce2449aba06ac11215b59788: Log is configured to *not* fsync() on all Append() calls
I20260811 19:02:48.309530 29922 log.cc:824] T 9768ba0ee6164761b54ed6fdcd7865b2 P fddff96ae4b84cdc86942277531be082: Log is configured to *not* fsync() on all Append() calls
I20260811 19:02:48.312024 29922 tablet_bootstrap.cc:492] T 9768ba0ee6164761b54ed6fdcd7865b2 P fddff96ae4b84cdc86942277531be082: No bootstrap required, opened a new log
I20260811 19:02:48.312024 29921 tablet_bootstrap.cc:492] T 9768ba0ee6164761b54ed6fdcd7865b2 P 7f911943ce2449aba06ac11215b59788: No bootstrap required, opened a new log
I20260811 19:02:48.312258 29921 ts_tablet_manager.cc:1397] T 9768ba0ee6164761b54ed6fdcd7865b2 P 7f911943ce2449aba06ac11215b59788: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 19:02:48.312258 29922 ts_tablet_manager.cc:1397] T 9768ba0ee6164761b54ed6fdcd7865b2 P fddff96ae4b84cdc86942277531be082: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 19:02:48.316504 29921 raft_consensus.cc:348] T 9768ba0ee6164761b54ed6fdcd7865b2 P 7f911943ce2449aba06ac11215b59788 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fddff96ae4b84cdc86942277531be082" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34823 } } peers { permanent_uuid: "7f911943ce2449aba06ac11215b59788" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43983 } } peers { permanent_uuid: "a088e3d057b849d593f4d1972111e6c8" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 42453 } }
I20260811 19:02:48.316504 29922 raft_consensus.cc:348] T 9768ba0ee6164761b54ed6fdcd7865b2 P fddff96ae4b84cdc86942277531be082 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fddff96ae4b84cdc86942277531be082" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34823 } } peers { permanent_uuid: "7f911943ce2449aba06ac11215b59788" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43983 } } peers { permanent_uuid: "a088e3d057b849d593f4d1972111e6c8" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 42453 } }
I20260811 19:02:48.316844 29921 raft_consensus.cc:374] T 9768ba0ee6164761b54ed6fdcd7865b2 P 7f911943ce2449aba06ac11215b59788 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:02:48.316844 29922 raft_consensus.cc:374] T 9768ba0ee6164761b54ed6fdcd7865b2 P fddff96ae4b84cdc86942277531be082 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:02:48.316900 29921 raft_consensus.cc:729] T 9768ba0ee6164761b54ed6fdcd7865b2 P 7f911943ce2449aba06ac11215b59788 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f911943ce2449aba06ac11215b59788, State: Initialized, Role: FOLLOWER
I20260811 19:02:48.316900 29922 raft_consensus.cc:729] T 9768ba0ee6164761b54ed6fdcd7865b2 P fddff96ae4b84cdc86942277531be082 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fddff96ae4b84cdc86942277531be082, State: Initialized, Role: FOLLOWER
W20260811 19:02:48.317358 29607 tablet.cc:2369] T 9768ba0ee6164761b54ed6fdcd7865b2 P a088e3d057b849d593f4d1972111e6c8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 19:02:48.317448 29921 consensus_queue.cc:260] T 9768ba0ee6164761b54ed6fdcd7865b2 P 7f911943ce2449aba06ac11215b59788 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fddff96ae4b84cdc86942277531be082" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34823 } } peers { permanent_uuid: "7f911943ce2449aba06ac11215b59788" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43983 } } peers { permanent_uuid: "a088e3d057b849d593f4d1972111e6c8" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 42453 } }
I20260811 19:02:48.317448 29922 consensus_queue.cc:260] T 9768ba0ee6164761b54ed6fdcd7865b2 P fddff96ae4b84cdc86942277531be082 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fddff96ae4b84cdc86942277531be082" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34823 } } peers { permanent_uuid: "7f911943ce2449aba06ac11215b59788" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43983 } } peers { permanent_uuid: "a088e3d057b849d593f4d1972111e6c8" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 42453 } }
I20260811 19:02:48.318638 29922 ts_tablet_manager.cc:1428] T 9768ba0ee6164761b54ed6fdcd7865b2 P fddff96ae4b84cdc86942277531be082: Time spent starting tablet: real 0.006s user 0.004s sys 0.003s
I20260811 19:02:48.318638 29921 ts_tablet_manager.cc:1428] T 9768ba0ee6164761b54ed6fdcd7865b2 P 7f911943ce2449aba06ac11215b59788: Time spent starting tablet: real 0.006s user 0.004s sys 0.003s
I20260811 19:02:48.318673 29904 heartbeater.cc:503] Master 127.13.109.62:37073 was elected leader, sending a full tablet report...
I20260811 19:02:48.319088 29759 heartbeater.cc:503] Master 127.13.109.62:37073 was elected leader, sending a full tablet report...
I20260811 19:02:48.390782 29927 raft_consensus.cc:482] T 9768ba0ee6164761b54ed6fdcd7865b2 P 7f911943ce2449aba06ac11215b59788 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:02:48.391024 29927 raft_consensus.cc:504] T 9768ba0ee6164761b54ed6fdcd7865b2 P 7f911943ce2449aba06ac11215b59788 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fddff96ae4b84cdc86942277531be082" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34823 } } peers { permanent_uuid: "7f911943ce2449aba06ac11215b59788" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43983 } } peers { permanent_uuid: "a088e3d057b849d593f4d1972111e6c8" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 42453 } }
I20260811 19:02:48.392184 29927 leader_election.cc:288] T 9768ba0ee6164761b54ed6fdcd7865b2 P 7f911943ce2449aba06ac11215b59788 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fddff96ae4b84cdc86942277531be082 (127.13.109.2:34823), a088e3d057b849d593f4d1972111e6c8 (127.13.109.1:42453)
I20260811 19:02:48.396314 29906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:48.392548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.2:34823 (local address 127.13.109.3:56639)
0811 19:02:48.392686 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:48.392723 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 19:02:48.392836 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:02:48.393425 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:02:48.393428 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:02:48.393447 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:02:48.393569 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:48.393575 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:48.394357 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:48.394361 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:48.395186 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:48.395195 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:48.395853 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:48.395857 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:48.395965 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:48.395987 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 19:02:48.396083 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 19:02:48.396175 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50,"mutex_wait_us":25}
I20260811 19:02:48.396314 29916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:48.392651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.3:56639 (local address 127.13.109.2:34823)
0811 19:02:48.392809 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:48.392814 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:48.392830 (+ 16us) server_negotiation.cc:407] Connection header received
0811 19:02:48.393241 (+ 411us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:48.393245 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:48.393303 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:48.393404 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:02:48.393683 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:48.394248 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:48.395314 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:48.395733 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:48.395863 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:48.395918 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:48.396100 (+ 182us) server_negotiation.cc:299] Negotiation successful
0811 19:02:48.396175 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 19:02:48.396533 29929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:48.392870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.1:42453 (local address 127.13.109.3:46487)
0811 19:02:48.393242 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:48.393254 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 19:02:48.393337 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:02:48.393669 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:02:48.393671 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:02:48.393687 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:02:48.393864 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:48.393871 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:48.394604 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:48.394610 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:48.395396 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:48.395405 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:48.396123 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:48.396127 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:48.396223 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:48.396244 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 19:02:48.396304 (+ 60us) client_negotiation.cc:240] Negotiation successful
0811 19:02:48.396377 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"mutex_wait_us":1,"thread_start_us":167,"threads_started":1}
I20260811 19:02:48.396536 29930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:48.392998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.3:46487 (local address 127.13.109.1:42453)
0811 19:02:48.393277 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:48.393281 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:48.393297 (+ 16us) server_negotiation.cc:407] Connection header received
0811 19:02:48.393475 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:48.393479 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:48.393546 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:48.393660 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:02:48.393981 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:48.394489 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:48.395524 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:48.395992 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:48.396134 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:48.396314 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:48.396350 (+ 36us) server_negotiation.cc:299] Negotiation successful
0811 19:02:48.396420 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":107,"threads_started":1}
I20260811 19:02:48.397128 29556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9768ba0ee6164761b54ed6fdcd7865b2" candidate_uuid: "7f911943ce2449aba06ac11215b59788" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a088e3d057b849d593f4d1972111e6c8" is_pre_election: true
I20260811 19:02:48.397146 29704 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9768ba0ee6164761b54ed6fdcd7865b2" candidate_uuid: "7f911943ce2449aba06ac11215b59788" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fddff96ae4b84cdc86942277531be082" is_pre_election: true
I20260811 19:02:48.397478 29556 raft_consensus.cc:2454] T 9768ba0ee6164761b54ed6fdcd7865b2 P a088e3d057b849d593f4d1972111e6c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f911943ce2449aba06ac11215b59788 in term 0.
I20260811 19:02:48.397478 29704 raft_consensus.cc:2454] T 9768ba0ee6164761b54ed6fdcd7865b2 P fddff96ae4b84cdc86942277531be082 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f911943ce2449aba06ac11215b59788 in term 0.
I20260811 19:02:48.398097 29780 leader_election.cc:302] T 9768ba0ee6164761b54ed6fdcd7865b2 P 7f911943ce2449aba06ac11215b59788 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f911943ce2449aba06ac11215b59788, fddff96ae4b84cdc86942277531be082; no voters:
I20260811 19:02:48.398370 29927 raft_consensus.cc:2781] T 9768ba0ee6164761b54ed6fdcd7865b2 P 7f911943ce2449aba06ac11215b59788 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 19:02:48.398471 29927 raft_consensus.cc:482] T 9768ba0ee6164761b54ed6fdcd7865b2 P 7f911943ce2449aba06ac11215b59788 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 19:02:48.398550 29927 raft_consensus.cc:3037] T 9768ba0ee6164761b54ed6fdcd7865b2 P 7f911943ce2449aba06ac11215b59788 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:02:48.399796 29927 raft_consensus.cc:504] T 9768ba0ee6164761b54ed6fdcd7865b2 P 7f911943ce2449aba06ac11215b59788 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fddff96ae4b84cdc86942277531be082" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34823 } } peers { permanent_uuid: "7f911943ce2449aba06ac11215b59788" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43983 } } peers { permanent_uuid: "a088e3d057b849d593f4d1972111e6c8" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 42453 } }
I20260811 19:02:48.400187 29927 leader_election.cc:288] T 9768ba0ee6164761b54ed6fdcd7865b2 P 7f911943ce2449aba06ac11215b59788 [CANDIDATE]: Term 1 election: Requested vote from peers fddff96ae4b84cdc86942277531be082 (127.13.109.2:34823), a088e3d057b849d593f4d1972111e6c8 (127.13.109.1:42453)
I20260811 19:02:48.400628 29704 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9768ba0ee6164761b54ed6fdcd7865b2" candidate_uuid: "7f911943ce2449aba06ac11215b59788" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fddff96ae4b84cdc86942277531be082"
I20260811 19:02:48.400676 29556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9768ba0ee6164761b54ed6fdcd7865b2" candidate_uuid: "7f911943ce2449aba06ac11215b59788" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a088e3d057b849d593f4d1972111e6c8"
I20260811 19:02:48.400801 29556 raft_consensus.cc:3037] T 9768ba0ee6164761b54ed6fdcd7865b2 P a088e3d057b849d593f4d1972111e6c8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:02:48.400782 29704 raft_consensus.cc:3037] T 9768ba0ee6164761b54ed6fdcd7865b2 P fddff96ae4b84cdc86942277531be082 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:02:48.402231 29556 raft_consensus.cc:2454] T 9768ba0ee6164761b54ed6fdcd7865b2 P a088e3d057b849d593f4d1972111e6c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f911943ce2449aba06ac11215b59788 in term 1.
I20260811 19:02:48.402231 29704 raft_consensus.cc:2454] T 9768ba0ee6164761b54ed6fdcd7865b2 P fddff96ae4b84cdc86942277531be082 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f911943ce2449aba06ac11215b59788 in term 1.
I20260811 19:02:48.402555 29780 leader_election.cc:302] T 9768ba0ee6164761b54ed6fdcd7865b2 P 7f911943ce2449aba06ac11215b59788 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f911943ce2449aba06ac11215b59788, a088e3d057b849d593f4d1972111e6c8; no voters:
I20260811 19:02:48.402796 29927 raft_consensus.cc:2781] T 9768ba0ee6164761b54ed6fdcd7865b2 P 7f911943ce2449aba06ac11215b59788 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:02:48.403074 29927 raft_consensus.cc:686] T 9768ba0ee6164761b54ed6fdcd7865b2 P 7f911943ce2449aba06ac11215b59788 [term 1 LEADER]: Becoming Leader. State: Replica: 7f911943ce2449aba06ac11215b59788, State: Running, Role: LEADER
I20260811 19:02:48.403451 29927 consensus_queue.cc:237] T 9768ba0ee6164761b54ed6fdcd7865b2 P 7f911943ce2449aba06ac11215b59788 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fddff96ae4b84cdc86942277531be082" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34823 } } peers { permanent_uuid: "7f911943ce2449aba06ac11215b59788" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43983 } } peers { permanent_uuid: "a088e3d057b849d593f4d1972111e6c8" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 42453 } }
I20260811 19:02:48.407069 29400 catalog_manager.cc:5495] T 9768ba0ee6164761b54ed6fdcd7865b2 P 7f911943ce2449aba06ac11215b59788 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7f911943ce2449aba06ac11215b59788 (127.13.109.3). New cstate: current_term: 1 leader_uuid: "7f911943ce2449aba06ac11215b59788" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fddff96ae4b84cdc86942277531be082" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34823 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7f911943ce2449aba06ac11215b59788" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a088e3d057b849d593f4d1972111e6c8" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 42453 } health_report { overall_health: UNKNOWN } } }
I20260811 19:02:48.419623 29917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:48.415897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54918 (local address 127.13.109.3:43983)
0811 19:02:48.416047 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:48.416051 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:48.416066 (+ 15us) server_negotiation.cc:407] Connection header received
0811 19:02:48.416218 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:48.416220 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:48.416280 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:48.416395 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:02:48.416646 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:48.417167 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:48.418040 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:48.418261 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:48.418371 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:48.418700 (+ 329us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:02:48.419178 (+ 478us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:02:48.419318 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:48.419381 (+ 63us) server_negotiation.cc:299] Negotiation successful
0811 19:02:48.419449 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 19:02:48.425907 29704 raft_consensus.cc:1261] T 9768ba0ee6164761b54ed6fdcd7865b2 P fddff96ae4b84cdc86942277531be082 [term 1 FOLLOWER]: Refusing update from remote peer 7f911943ce2449aba06ac11215b59788: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:02:48.425904 29556 raft_consensus.cc:1261] T 9768ba0ee6164761b54ed6fdcd7865b2 P a088e3d057b849d593f4d1972111e6c8 [term 1 FOLLOWER]: Refusing update from remote peer 7f911943ce2449aba06ac11215b59788: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:02:48.426627 29927 consensus_queue.cc:1035] T 9768ba0ee6164761b54ed6fdcd7865b2 P 7f911943ce2449aba06ac11215b59788 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a088e3d057b849d593f4d1972111e6c8" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 42453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:02:48.426818 29931 consensus_queue.cc:1035] T 9768ba0ee6164761b54ed6fdcd7865b2 P 7f911943ce2449aba06ac11215b59788 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fddff96ae4b84cdc86942277531be082" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34823 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:02:48.434492 29936 mvcc.cc:205] Tried to move back new op lower bound from 7317401470664523776 to 7317401470583697408. Current Snapshot: MvccSnapshot[applied={T|T < 7317401470664523776}]
I20260811 19:02:48.434836 29935 mvcc.cc:205] Tried to move back new op lower bound from 7317401470664523776 to 7317401470583697408. Current Snapshot: MvccSnapshot[applied={T|T < 7317401470664523776}]
I20260811 19:02:48.443624 29401 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:36142:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:02:48.443840 29401 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:36142:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:02:48.446511 29401 catalog_manager.cc:5782] T 00000000000000000000000000000000 P df03a25279794bc29ae9e20f2d28c530: Sending DeleteTablet for 3 replicas of tablet 9768ba0ee6164761b54ed6fdcd7865b2
I20260811 19:02:48.447293 29535 tablet_service.cc:1514] Processing DeleteTablet for tablet 9768ba0ee6164761b54ed6fdcd7865b2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:02:48 UTC) from {username='kudu', principal='kudu/127.13.109.62@KRBTEST.COM'} at 127.0.0.1:40018
I20260811 19:02:48.447412 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 29441
I20260811 19:02:48.447405 29835 tablet_service.cc:1514] Processing DeleteTablet for tablet 9768ba0ee6164761b54ed6fdcd7865b2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:02:48 UTC) from {username='kudu', principal='kudu/127.13.109.62@KRBTEST.COM'} at 127.0.0.1:54908
I20260811 19:02:48.449051 29682 tablet_service.cc:1514] Processing DeleteTablet for tablet 9768ba0ee6164761b54ed6fdcd7865b2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:02:48 UTC) from {username='kudu', principal='kudu/127.13.109.62@KRBTEST.COM'} at 127.0.0.1:50912
I20260811 19:02:48.449658 29946 tablet_replica.cc:330] T 9768ba0ee6164761b54ed6fdcd7865b2 P fddff96ae4b84cdc86942277531be082: stopping tablet replica
I20260811 19:02:48.450075 29945 tablet_replica.cc:330] T 9768ba0ee6164761b54ed6fdcd7865b2 P 7f911943ce2449aba06ac11215b59788: stopping tablet replica
I20260811 19:02:48.450071 29946 raft_consensus.cc:2229] T 9768ba0ee6164761b54ed6fdcd7865b2 P fddff96ae4b84cdc86942277531be082 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:02:48.450444 29946 raft_consensus.cc:2258] T 9768ba0ee6164761b54ed6fdcd7865b2 P fddff96ae4b84cdc86942277531be082 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:02:48.450452 29945 raft_consensus.cc:2229] T 9768ba0ee6164761b54ed6fdcd7865b2 P 7f911943ce2449aba06ac11215b59788 [term 1 LEADER]: Raft consensus shutting down.
I20260811 19:02:48.451538 29945 raft_consensus.cc:2258] T 9768ba0ee6164761b54ed6fdcd7865b2 P 7f911943ce2449aba06ac11215b59788 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:02:48.452152 29946 ts_tablet_manager.cc:1905] T 9768ba0ee6164761b54ed6fdcd7865b2 P fddff96ae4b84cdc86942277531be082: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:02:48.453174 29945 ts_tablet_manager.cc:1905] T 9768ba0ee6164761b54ed6fdcd7865b2 P 7f911943ce2449aba06ac11215b59788: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:02:48.455998 29946 ts_tablet_manager.cc:1918] T 9768ba0ee6164761b54ed6fdcd7865b2 P fddff96ae4b84cdc86942277531be082: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:02:48.456019 29945 ts_tablet_manager.cc:1918] T 9768ba0ee6164761b54ed6fdcd7865b2 P 7f911943ce2449aba06ac11215b59788: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:02:48.456139 29946 log.cc:1196] T 9768ba0ee6164761b54ed6fdcd7865b2 P fddff96ae4b84cdc86942277531be082: Deleting WAL directory at /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-1/wal/wals/9768ba0ee6164761b54ed6fdcd7865b2
I20260811 19:02:48.456271 29945 log.cc:1196] T 9768ba0ee6164761b54ed6fdcd7865b2 P 7f911943ce2449aba06ac11215b59788: Deleting WAL directory at /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-2/wal/wals/9768ba0ee6164761b54ed6fdcd7865b2
I20260811 19:02:48.456508 29946 ts_tablet_manager.cc:1939] T 9768ba0ee6164761b54ed6fdcd7865b2 P fddff96ae4b84cdc86942277531be082: Deleting consensus metadata
I20260811 19:02:48.456790 29945 ts_tablet_manager.cc:1939] T 9768ba0ee6164761b54ed6fdcd7865b2 P 7f911943ce2449aba06ac11215b59788: Deleting consensus metadata
I20260811 19:02:48.458122 29384 catalog_manager.cc:4841] TS 7f911943ce2449aba06ac11215b59788 (127.13.109.3:43983): tablet 9768ba0ee6164761b54ed6fdcd7865b2 (table test-table [id=bfed1122de8341f1bcff80f7e70a6af1]) successfully deleted
I20260811 19:02:48.458125 29383 catalog_manager.cc:4841] TS fddff96ae4b84cdc86942277531be082 (127.13.109.2:34823): tablet 9768ba0ee6164761b54ed6fdcd7865b2 (table test-table [id=bfed1122de8341f1bcff80f7e70a6af1]) successfully deleted
W20260811 19:02:48.460235 29383 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.109.1:42453 (error 108)
I20260811 19:02:48.460856 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 29611
I20260811 19:02:48.462221 29915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:48.461087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.1:42453 (local address 127.0.0.1:40020)
0811 19:02:48.462027 (+ 940us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:48.462109 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.1:42453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":822}
W20260811 19:02:48.462807 29383 catalog_manager.cc:4584] TS a088e3d057b849d593f4d1972111e6c8 (127.13.109.1:42453): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9768ba0ee6164761b54ed6fdcd7865b2: Network error: Client connection negotiation failed: client connection to 127.13.109.1:42453: connect: Connection refused (error 111)
I20260811 19:02:48.468863 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 29762
I20260811 19:02:48.476723 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 29365
I20260811 19:02:48.486301 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.62:37073
--webserver_interface=127.13.109.62
--webserver_port=39633
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab
--principal=kudu/127.13.109.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:38669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.109.62:37073
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:02:48.608489 29948 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:02:48.608820 29948 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:02:48.608866 29948 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:02:48.613209 29948 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:02:48.613312 29948 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:02:48.613333 29948 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:02:48.613351 29948 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:02:48.613368 29948 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:02:48.619444 29948 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:38669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.109.62:37073
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab
--principal=kudu/127.13.109.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.62:37073
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.109.62
--webserver_port=39633
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29948
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:02:48.620764 29948 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:02:48.621934 29948 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:02:48.631033 29956 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:02:48.631217 29954 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:02:48.631033 29953 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:02:48.632283 29948 server_base.cc:1034] running on GCE node
I20260811 19:02:48.632939 29948 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:02:48.634207 29948 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:02:48.635427 29948 hybrid_clock.cc:648] HybridClock initialized: now 1786474968635402 us; error 38 us; skew 500 ppm
Aug 11 19:02:48 dist-test-slave-0kls krb5kdc[28748](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474968, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:02:48.638729 29948 init.cc:377] Logged in from keytab as kudu/127.13.109.62@KRBTEST.COM (short username kudu)
I20260811 19:02:48.639894 29948 webserver.cc:460] Webserver started at http://127.13.109.62:39633/ using document root <none> and password file <none>
I20260811 19:02:48.640543 29948 fs_manager.cc:359] Metadata directory not provided
I20260811 19:02:48.640646 29948 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:02:48.645102 29948 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 19:02:48.648303 29964 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:48.649961 29948 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 19:02:48.650158 29948 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/master-0/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/master-0/wal
uuid: "df03a25279794bc29ae9e20f2d28c530"
format_stamp: "Formatted at 2026-08-11 19:02:46 on dist-test-slave-0kls"
server_key: "f63b3ab253bfa165821434b618940ab1"
server_key_iv: "b4834185cf87588d47aea14f01621ed8"
server_key_version: "encryptionkey@0"
I20260811 19:02:48.650816 29948 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:02:48.666465 29948 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:02:48.670288 29948 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:02:48.670538 29948 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:02:48.678748 29948 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.62:37073
I20260811 19:02:48.679085 30023 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.62:37073 every 8 connection(s)
I20260811 19:02:48.680006 29948 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
I20260811 19:02:48.685026 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 29948
I20260811 19:02:48.685115 30024 sys_catalog.cc:263] Verifying existing consensus state
I20260811 19:02:48.685583 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.1:42453
--local_ip_for_outbound_sockets=127.13.109.1
--tserver_master_addrs=127.13.109.62:37073
--webserver_port=46621
--webserver_interface=127.13.109.1
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.1.keytab
--principal=kudu/127.13.109.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:38669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 19:02:48.687932 30024 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df03a25279794bc29ae9e20f2d28c530: Bootstrap starting.
I20260811 19:02:48.707495 30024 log.cc:824] T 00000000000000000000000000000000 P df03a25279794bc29ae9e20f2d28c530: Log is configured to *not* fsync() on all Append() calls
I20260811 19:02:48.720314 30024 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df03a25279794bc29ae9e20f2d28c530: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 19:02:48.720806 30024 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df03a25279794bc29ae9e20f2d28c530: Bootstrap complete.
I20260811 19:02:48.725286 30024 raft_consensus.cc:348] T 00000000000000000000000000000000 P df03a25279794bc29ae9e20f2d28c530 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df03a25279794bc29ae9e20f2d28c530" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 37073 } }
I20260811 19:02:48.725721 30024 raft_consensus.cc:729] T 00000000000000000000000000000000 P df03a25279794bc29ae9e20f2d28c530 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: df03a25279794bc29ae9e20f2d28c530, State: Initialized, Role: FOLLOWER
I20260811 19:02:48.726352 30024 consensus_queue.cc:260] T 00000000000000000000000000000000 P df03a25279794bc29ae9e20f2d28c530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df03a25279794bc29ae9e20f2d28c530" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 37073 } }
I20260811 19:02:48.726480 30024 raft_consensus.cc:388] T 00000000000000000000000000000000 P df03a25279794bc29ae9e20f2d28c530 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:02:48.726577 30024 raft_consensus.cc:482] T 00000000000000000000000000000000 P df03a25279794bc29ae9e20f2d28c530 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:02:48.726732 30024 raft_consensus.cc:3037] T 00000000000000000000000000000000 P df03a25279794bc29ae9e20f2d28c530 [term 2 FOLLOWER]: Advancing to term 3
I20260811 19:02:48.728864 30024 raft_consensus.cc:504] T 00000000000000000000000000000000 P df03a25279794bc29ae9e20f2d28c530 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df03a25279794bc29ae9e20f2d28c530" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 37073 } }
I20260811 19:02:48.729485 30024 leader_election.cc:302] T 00000000000000000000000000000000 P df03a25279794bc29ae9e20f2d28c530 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: df03a25279794bc29ae9e20f2d28c530; no voters:
I20260811 19:02:48.729950 30024 leader_election.cc:288] T 00000000000000000000000000000000 P df03a25279794bc29ae9e20f2d28c530 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 19:02:48.730053 30029 raft_consensus.cc:2781] T 00000000000000000000000000000000 P df03a25279794bc29ae9e20f2d28c530 [term 3 FOLLOWER]: Leader election won for term 3
I20260811 19:02:48.730294 30029 raft_consensus.cc:686] T 00000000000000000000000000000000 P df03a25279794bc29ae9e20f2d28c530 [term 3 LEADER]: Becoming Leader. State: Replica: df03a25279794bc29ae9e20f2d28c530, State: Running, Role: LEADER
I20260811 19:02:48.730641 30029 consensus_queue.cc:237] T 00000000000000000000000000000000 P df03a25279794bc29ae9e20f2d28c530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df03a25279794bc29ae9e20f2d28c530" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 37073 } }
I20260811 19:02:48.731577 30024 sys_catalog.cc:564] T 00000000000000000000000000000000 P df03a25279794bc29ae9e20f2d28c530 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:02:48.732445 30031 sys_catalog.cc:455] T 00000000000000000000000000000000 P df03a25279794bc29ae9e20f2d28c530 [sys.catalog]: SysCatalogTable state changed. Reason: New leader df03a25279794bc29ae9e20f2d28c530. Latest consensus state: current_term: 3 leader_uuid: "df03a25279794bc29ae9e20f2d28c530" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df03a25279794bc29ae9e20f2d28c530" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 37073 } } }
I20260811 19:02:48.732499 30030 sys_catalog.cc:455] T 00000000000000000000000000000000 P df03a25279794bc29ae9e20f2d28c530 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "df03a25279794bc29ae9e20f2d28c530" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df03a25279794bc29ae9e20f2d28c530" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 37073 } } }
I20260811 19:02:48.732646 30030 sys_catalog.cc:458] T 00000000000000000000000000000000 P df03a25279794bc29ae9e20f2d28c530 [sys.catalog]: This master's current role is: LEADER
I20260811 19:02:48.732594 30031 sys_catalog.cc:458] T 00000000000000000000000000000000 P df03a25279794bc29ae9e20f2d28c530 [sys.catalog]: This master's current role is: LEADER
I20260811 19:02:48.733049 30038 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:02:48.739223 30038 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:02:48.740298 30038 catalog_manager.cc:1259] Loaded cluster ID: 77858b6acb72403bb610ad5d3895ea9a
I20260811 19:02:48.740383 30038 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260811 19:02:48.741662 30038 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f8745fe79ad google::LogMessage::Fail() at ??:0
@ 0x7f8745fea88a google::LogMessage::SendToLog() at ??:0
@ 0x7f8745fe74de google::LogMessage::Flush() at ??:0
@ 0x7f8745fe8a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f8750926fdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f8750927d2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f8750924579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f8750968640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f87482393d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f8748239c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f874823b3f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f874822a512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f874986e6db start_thread at ??:0
@ 0x7f8744ae561f clone at ??:0
W20260811 19:02:48.846975 30026 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:02:48.847260 30026 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:02:48.847311 30026 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:02:48.851526 30026 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:02:48.851598 30026 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:02:48.851662 30026 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.1
I20260811 19:02:48.857290 30026 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:38669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.1.keytab
--principal=kudu/127.13.109.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.1:42453
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.109.1
--webserver_port=46621
--tserver_master_addrs=127.13.109.62:37073
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30026
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.1
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:02:48.858623 30026 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:02:48.859719 30026 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:02:48.871953 30072 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:02:48.871981 30068 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:02:48.872047 30069 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:02:48.872306 30026 server_base.cc:1034] running on GCE node
I20260811 19:02:48.872990 30026 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:02:48.874061 30026 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:02:48.875317 30026 hybrid_clock.cc:648] HybridClock initialized: now 1786474968875272 us; error 59 us; skew 500 ppm
Aug 11 19:02:48 dist-test-slave-0kls krb5kdc[28748](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474968, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:02:48.879384 30026 init.cc:377] Logged in from keytab as kudu/127.13.109.1@KRBTEST.COM (short username kudu)
I20260811 19:02:48.880790 30026 webserver.cc:460] Webserver started at http://127.13.109.1:46621/ using document root <none> and password file <none>
I20260811 19:02:48.881752 30026 fs_manager.cc:359] Metadata directory not provided
I20260811 19:02:48.881896 30026 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:02:48.887658 30026 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 19:02:48.890430 30079 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:48.892021 30026 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 19:02:48.892220 30026 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-0/wal
uuid: "a088e3d057b849d593f4d1972111e6c8"
format_stamp: "Formatted at 2026-08-11 19:02:46 on dist-test-slave-0kls"
server_key: "ef85a7a5af195b603801d7ede48c299e"
server_key_iv: "2dd632a0f7272b236484ff4c805f4ce1"
server_key_version: "encryptionkey@0"
I20260811 19:02:48.892871 30026 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:02:48.921305 30026 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:02:48.925246 30026 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:02:48.925580 30026 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:02:48.926367 30026 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:02:48.927770 30086 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 19:02:48.934186 30026 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 19:02:48.934309 30026 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.007s user 0.001s sys 0.000s
I20260811 19:02:48.934417 30026 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 19:02:48.936319 30026 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 19:02:48.936415 30026 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260811 19:02:48.936789 30086 tablet_bootstrap.cc:492] T 9768ba0ee6164761b54ed6fdcd7865b2 P a088e3d057b849d593f4d1972111e6c8: Bootstrap starting.
I20260811 19:02:48.948261 30026 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.1:42453
I20260811 19:02:48.948580 30203 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.1:42453 every 8 connection(s)
I20260811 19:02:48.949465 30026 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
I20260811 19:02:48.957886 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 30026
I20260811 19:02:48.958424 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.2:34823
--local_ip_for_outbound_sockets=127.13.109.2
--tserver_master_addrs=127.13.109.62:37073
--webserver_port=38343
--webserver_interface=127.13.109.2
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.2.keytab
--principal=kudu/127.13.109.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:38669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 19:02:48.958734 30086 log.cc:824] T 9768ba0ee6164761b54ed6fdcd7865b2 P a088e3d057b849d593f4d1972111e6c8: Log is configured to *not* fsync() on all Append() calls
I20260811 19:02:48.967144 30086 tablet_bootstrap.cc:492] T 9768ba0ee6164761b54ed6fdcd7865b2 P a088e3d057b849d593f4d1972111e6c8: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 19:02:48.967600 30086 tablet_bootstrap.cc:492] T 9768ba0ee6164761b54ed6fdcd7865b2 P a088e3d057b849d593f4d1972111e6c8: Bootstrap complete.
I20260811 19:02:48.975715 30086 ts_tablet_manager.cc:1397] T 9768ba0ee6164761b54ed6fdcd7865b2 P a088e3d057b849d593f4d1972111e6c8: Time spent bootstrapping tablet: real 0.039s user 0.023s sys 0.007s
I20260811 19:02:48.979391 30086 raft_consensus.cc:348] T 9768ba0ee6164761b54ed6fdcd7865b2 P a088e3d057b849d593f4d1972111e6c8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fddff96ae4b84cdc86942277531be082" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34823 } } peers { permanent_uuid: "7f911943ce2449aba06ac11215b59788" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43983 } } peers { permanent_uuid: "a088e3d057b849d593f4d1972111e6c8" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 42453 } }
I20260811 19:02:48.980013 30086 raft_consensus.cc:729] T 9768ba0ee6164761b54ed6fdcd7865b2 P a088e3d057b849d593f4d1972111e6c8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a088e3d057b849d593f4d1972111e6c8, State: Initialized, Role: FOLLOWER
I20260811 19:02:48.980597 30086 consensus_queue.cc:260] T 9768ba0ee6164761b54ed6fdcd7865b2 P a088e3d057b849d593f4d1972111e6c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fddff96ae4b84cdc86942277531be082" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34823 } } peers { permanent_uuid: "7f911943ce2449aba06ac11215b59788" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43983 } } peers { permanent_uuid: "a088e3d057b849d593f4d1972111e6c8" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 42453 } }
I20260811 19:02:48.981833 30086 ts_tablet_manager.cc:1428] T 9768ba0ee6164761b54ed6fdcd7865b2 P a088e3d057b849d593f4d1972111e6c8: Time spent starting tablet: real 0.006s user 0.007s sys 0.001s
W20260811 19:02:48.982887 30206 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:02:48.952166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:37073 (local address 127.13.109.1:34947)
0811 19:02:48.952846 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:48.952911 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0811 19:02:48.954252 (+ 1341us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:02:48.982137 (+ 27885us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:37073: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":180,"threads_started":1}
W20260811 19:02:48.983453 30204 heartbeater.cc:644] Failed to heartbeat to 127.13.109.62:37073 (0 consecutive failures): Network error: Failed to ping master at 127.13.109.62:37073: Client connection negotiation failed: client connection to 127.13.109.62:37073: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 19:02:48.984186 30206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:48.983924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:37073 (local address 127.13.109.1:50233)
0811 19:02:48.984056 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:48.984130 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:37073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260811 19:02:48.984828 30206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:48.984614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:37073 (local address 127.13.109.1:44713)
0811 19:02:48.984724 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:48.984771 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:37073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 19:02:48.984984 30204 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 19:02:49.091511 30208 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:02:49.091895 30208 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:02:49.091989 30208 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:02:49.096452 30208 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:02:49.096542 30208 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:02:49.096609 30208 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.2
I20260811 19:02:49.103156 30208 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:38669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.2.keytab
--principal=kudu/127.13.109.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.2:34823
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.109.2
--webserver_port=38343
--tserver_master_addrs=127.13.109.62:37073
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30208
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.2
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:02:49.104420 30208 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:02:49.105504 30208 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:02:49.116353 30216 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:02:49.116357 30219 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:02:49.116357 30214 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:02:49.116583 30208 server_base.cc:1034] running on GCE node
I20260811 19:02:49.117249 30208 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:02:49.118033 30208 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:02:49.119207 30208 hybrid_clock.cc:648] HybridClock initialized: now 1786474969119189 us; error 32 us; skew 500 ppm
Aug 11 19:02:49 dist-test-slave-0kls krb5kdc[28748](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474969, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:02:49.122517 30208 init.cc:377] Logged in from keytab as kudu/127.13.109.2@KRBTEST.COM (short username kudu)
I20260811 19:02:49.123610 30208 webserver.cc:460] Webserver started at http://127.13.109.2:38343/ using document root <none> and password file <none>
I20260811 19:02:49.124303 30208 fs_manager.cc:359] Metadata directory not provided
I20260811 19:02:49.124415 30208 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:02:49.129228 30208 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 19:02:49.131917 30226 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:49.133455 30208 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 19:02:49.133661 30208 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-1/wal
uuid: "fddff96ae4b84cdc86942277531be082"
format_stamp: "Formatted at 2026-08-11 19:02:46 on dist-test-slave-0kls"
server_key: "75888febf87c85be2459138e377876ca"
server_key_iv: "d05678c0d1c93a01a233a1475e19d331"
server_key_version: "encryptionkey@0"
I20260811 19:02:49.134404 30208 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:02:49.147972 30208 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:02:49.151620 30208 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:02:49.151856 30208 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:02:49.152522 30208 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:02:49.153487 30208 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:02:49.153560 30208 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:49.153635 30208 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:02:49.153685 30208 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:49.163666 30208 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.2:34823
I20260811 19:02:49.163684 30348 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.2:34823 every 8 connection(s)
I20260811 19:02:49.164757 30208 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
I20260811 19:02:49.168802 30352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:49.167105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:37073 (local address 127.13.109.2:42525)
0811 19:02:49.167608 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:49.168047 (+ 439us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:37073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":111,"threads_started":1}
W20260811 19:02:49.169282 30349 heartbeater.cc:644] Failed to heartbeat to 127.13.109.62:37073 (0 consecutive failures): Network error: Failed to ping master at 127.13.109.62:37073: Client connection negotiation failed: client connection to 127.13.109.62:37073: connect: Connection refused (error 111)
I20260811 19:02:49.170030 30352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:49.169750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:37073 (local address 127.13.109.2:39353)
0811 19:02:49.169903 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:49.169966 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:37073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260811 19:02:49.170687 30352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:49.170449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:37073 (local address 127.13.109.2:56601)
0811 19:02:49.170583 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:49.170635 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:37073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260811 19:02:49.170871 30349 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 19:02:49.173710 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 30208
I20260811 19:02:49.174216 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.3:43983
--local_ip_for_outbound_sockets=127.13.109.3
--tserver_master_addrs=127.13.109.62:37073
--webserver_port=39733
--webserver_interface=127.13.109.3
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.3.keytab
--principal=kudu/127.13.109.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:38669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:02:49.297302 30353 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:02:49.297658 30353 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:02:49.297745 30353 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:02:49.302206 30353 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:02:49.302281 30353 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:02:49.302378 30353 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.3
I20260811 19:02:49.308368 30353 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:38669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.3.keytab
--principal=kudu/127.13.109.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.3:43983
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.109.3
--webserver_port=39733
--tserver_master_addrs=127.13.109.62:37073
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30353
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.3
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:02:49.309729 30353 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:02:49.311056 30353 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:02:49.322649 30365 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:02:49.322678 30353 server_base.cc:1034] running on GCE node
W20260811 19:02:49.322649 30359 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:02:49.322721 30360 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:02:49.323500 30353 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:02:49.324345 30353 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:02:49.325615 30353 hybrid_clock.cc:648] HybridClock initialized: now 1786474969325567 us; error 54 us; skew 500 ppm
Aug 11 19:02:49 dist-test-slave-0kls krb5kdc[28748](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474969, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:02:49.329074 30353 init.cc:377] Logged in from keytab as kudu/127.13.109.3@KRBTEST.COM (short username kudu)
I20260811 19:02:49.330289 30353 webserver.cc:460] Webserver started at http://127.13.109.3:39733/ using document root <none> and password file <none>
I20260811 19:02:49.331014 30353 fs_manager.cc:359] Metadata directory not provided
I20260811 19:02:49.331103 30353 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:02:49.336181 30353 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 19:02:49.339267 30372 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:49.340921 30353 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 19:02:49.341153 30353 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-2/wal
uuid: "7f911943ce2449aba06ac11215b59788"
format_stamp: "Formatted at 2026-08-11 19:02:47 on dist-test-slave-0kls"
server_key: "b58775d7cbbb98b1c1833ff3b12d2d18"
server_key_iv: "72f02612bbefbb73059d05dc5554a9f8"
server_key_version: "encryptionkey@0"
I20260811 19:02:49.341964 30353 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:02:49.355749 30353 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:02:49.359594 30353 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:02:49.359894 30353 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:02:49.360653 30353 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:02:49.361686 30353 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:02:49.361740 30353 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:49.361850 30353 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:02:49.361907 30353 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:49.372189 30353 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.3:43983
I20260811 19:02:49.372254 30501 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.3:43983 every 8 connection(s)
I20260811 19:02:49.373548 30353 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
I20260811 19:02:49.377985 30504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:49.376069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:37073 (local address 127.13.109.3:51029)
0811 19:02:49.376613 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:49.377088 (+ 475us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:37073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":134,"threads_started":1}
W20260811 19:02:49.378538 30502 heartbeater.cc:644] Failed to heartbeat to 127.13.109.62:37073 (0 consecutive failures): Network error: Failed to ping master at 127.13.109.62:37073: Client connection negotiation failed: client connection to 127.13.109.62:37073: connect: Connection refused (error 111)
I20260811 19:02:49.379256 30504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:49.379007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:37073 (local address 127.13.109.3:55069)
0811 19:02:49.379126 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:49.379196 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:37073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260811 19:02:49.380003 30504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:49.379804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:37073 (local address 127.13.109.3:35397)
0811 19:02:49.379909 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:49.379952 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:37073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260811 19:02:49.380230 30502 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 19:02:49.382910 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 30353
I20260811 19:02:49.986480 30506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:49.985938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:37073 (local address 127.13.109.1:38277)
0811 19:02:49.986283 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:49.986354 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:37073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":100,"threads_started":1}
I20260811 19:02:50.162563 30507 raft_consensus.cc:482] T 9768ba0ee6164761b54ed6fdcd7865b2 P a088e3d057b849d593f4d1972111e6c8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:02:50.162838 30507 raft_consensus.cc:504] T 9768ba0ee6164761b54ed6fdcd7865b2 P a088e3d057b849d593f4d1972111e6c8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fddff96ae4b84cdc86942277531be082" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34823 } } peers { permanent_uuid: "7f911943ce2449aba06ac11215b59788" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43983 } } peers { permanent_uuid: "a088e3d057b849d593f4d1972111e6c8" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 42453 } }
I20260811 19:02:50.164336 30507 leader_election.cc:288] T 9768ba0ee6164761b54ed6fdcd7865b2 P a088e3d057b849d593f4d1972111e6c8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fddff96ae4b84cdc86942277531be082 (127.13.109.2:34823), 7f911943ce2449aba06ac11215b59788 (127.13.109.3:43983)
I20260811 19:02:50.171936 30511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:50.171455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:37073 (local address 127.13.109.2:47563)
0811 19:02:50.171741 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:50.171806 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:37073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":105,"threads_started":1}
Aug 11 19:02:50 dist-test-slave-0kls krb5kdc[28748](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474968, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.1@KRBTEST.COM for kudu/127.13.109.3@KRBTEST.COM
Aug 11 19:02:50 dist-test-slave-0kls krb5kdc[28748](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474968, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.1@KRBTEST.COM for kudu/127.13.109.2@KRBTEST.COM
I20260811 19:02:50.180233 30508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:50.165018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.1:46765 (local address 127.13.109.2:34823)
0811 19:02:50.165802 (+ 784us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:50.165818 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:50.165866 (+ 48us) server_negotiation.cc:407] Connection header received
0811 19:02:50.166771 (+ 905us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:50.166780 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:50.167233 (+ 453us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:50.167544 (+ 311us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:02:50.168962 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:50.170089 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:50.171421 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:50.172028 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:50.172202 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:50.175510 (+ 3308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:02:50.175515 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:02:50.175526 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:02:50.175556 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:02:50.177303 (+ 1747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:50.177862 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:50.177871 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:50.177881 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:50.177996 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:50.178330 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:50.178335 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:50.178337 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:50.178905 (+ 568us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:02:50.179067 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:50.179843 (+ 776us) server_negotiation.cc:299] Negotiation successful
0811 19:02:50.180084 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":626,"thread_start_us":353,"threads_started":1}
I20260811 19:02:50.180337 30506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:50.164671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.3:43983 (local address 127.13.109.1:43567)
0811 19:02:50.164950 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:50.164973 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 19:02:50.165137 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:02:50.168112 (+ 2975us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:02:50.168115 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:02:50.168348 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:02:50.168766 (+ 418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:50.168782 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:50.170158 (+ 1376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:50.170162 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:50.171251 (+ 1089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:50.171261 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:50.172146 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:50.172150 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:50.172343 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:50.173021 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:02:50.173046 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:02:50.174842 (+ 1796us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:02:50.177329 (+ 2487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:02:50.177336 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:02:50.177350 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:02:50.177642 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:02:50.178038 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:02:50.178042 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:02:50.178044 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:02:50.178234 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:02:50.179074 (+ 840us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:02:50.179083 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:02:50.179461 (+ 378us) client_negotiation.cc:769] Sending connection context
0811 19:02:50.179693 (+ 232us) client_negotiation.cc:240] Negotiation successful
0811 19:02:50.180174 (+ 481us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"mutex_wait_us":265,"spinlock_wait_cycles":3200}
I20260811 19:02:50.180336 30509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:50.164678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.2:34823 (local address 127.13.109.1:46765)
0811 19:02:50.165751 (+ 1073us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:50.165768 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 19:02:50.165893 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:02:50.167663 (+ 1770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:02:50.167670 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:02:50.168234 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:02:50.168769 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:50.168785 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:50.170249 (+ 1464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:50.170253 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:50.171258 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:50.171273 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:50.172398 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:50.172402 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:50.172504 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:50.173146 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:02:50.173159 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:02:50.175278 (+ 2119us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:02:50.177439 (+ 2161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:02:50.177445 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:02:50.177448 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:02:50.177719 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:02:50.178099 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:02:50.178104 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:02:50.178106 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:02:50.178222 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:02:50.179113 (+ 891us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:02:50.179115 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:02:50.179461 (+ 346us) client_negotiation.cc:769] Sending connection context
0811 19:02:50.179693 (+ 232us) client_negotiation.cc:240] Negotiation successful
0811 19:02:50.180126 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":996,"mutex_wait_us":793,"spinlock_wait_cycles":11520,"thread_start_us":123,"threads_started":1}
I20260811 19:02:50.181068 30510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:50.165416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.1:43567 (local address 127.13.109.3:43983)
0811 19:02:50.165914 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:50.165925 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:50.165957 (+ 32us) server_negotiation.cc:407] Connection header received
0811 19:02:50.166773 (+ 816us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:50.166780 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:50.167217 (+ 437us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:50.168168 (+ 951us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:02:50.168938 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:50.170002 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:50.171378 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:50.172030 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:50.172200 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:50.175072 (+ 2872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:02:50.175078 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:02:50.175091 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:02:50.175121 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:02:50.177195 (+ 2074us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:50.177772 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:50.177779 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:50.177805 (+ 26us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:50.177937 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:50.178353 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:50.178358 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:50.178361 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:50.178903 (+ 542us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:02:50.179065 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:50.180599 (+ 1534us) server_negotiation.cc:299] Negotiation successful
0811 19:02:50.180885 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":137,"threads_started":1}
W20260811 19:02:50.183499 30080 leader_election.cc:341] T 9768ba0ee6164761b54ed6fdcd7865b2 P a088e3d057b849d593f4d1972111e6c8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fddff96ae4b84cdc86942277531be082 (127.13.109.2:34823): Not found: Tablet not found: 9768ba0ee6164761b54ed6fdcd7865b2
W20260811 19:02:50.183669 30081 leader_election.cc:341] T 9768ba0ee6164761b54ed6fdcd7865b2 P a088e3d057b849d593f4d1972111e6c8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7f911943ce2449aba06ac11215b59788 (127.13.109.3:43983): Not found: Tablet not found: 9768ba0ee6164761b54ed6fdcd7865b2
I20260811 19:02:50.183741 30080 leader_election.cc:302] T 9768ba0ee6164761b54ed6fdcd7865b2 P a088e3d057b849d593f4d1972111e6c8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a088e3d057b849d593f4d1972111e6c8; no voters: 7f911943ce2449aba06ac11215b59788, fddff96ae4b84cdc86942277531be082
I20260811 19:02:50.184058 30507 raft_consensus.cc:2726] T 9768ba0ee6164761b54ed6fdcd7865b2 P a088e3d057b849d593f4d1972111e6c8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 19:02:50.381239 30512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:50.380809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:37073 (local address 127.13.109.3:53327)
0811 19:02:50.381068 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:50.381120 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:37073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":79,"threads_started":1}
I20260811 19:02:50.987983 30513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:50.987491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:37073 (local address 127.13.109.1:52827)
0811 19:02:50.987804 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:50.987878 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:37073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":99,"threads_started":1}
I20260811 19:02:51.173722 30514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:51.173214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:37073 (local address 127.13.109.2:47649)
0811 19:02:51.173536 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:51.173612 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:37073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":88,"threads_started":1}
I20260811 19:02:51.382483 30518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:51.382022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:37073 (local address 127.13.109.3:57981)
0811 19:02:51.382305 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:51.382368 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:37073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":78,"threads_started":1}
I20260811 19:02:51.989569 30519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:51.989046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:37073 (local address 127.13.109.1:51837)
0811 19:02:51.989383 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:51.989452 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:37073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":82,"threads_started":1}
I20260811 19:02:52.175200 30520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:52.174682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:37073 (local address 127.13.109.2:45445)
0811 19:02:52.174998 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:52.175069 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:37073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":93,"threads_started":1}
I20260811 19:02:52.354807 30521 raft_consensus.cc:482] T 9768ba0ee6164761b54ed6fdcd7865b2 P a088e3d057b849d593f4d1972111e6c8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:02:52.354980 30521 raft_consensus.cc:504] T 9768ba0ee6164761b54ed6fdcd7865b2 P a088e3d057b849d593f4d1972111e6c8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fddff96ae4b84cdc86942277531be082" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34823 } } peers { permanent_uuid: "7f911943ce2449aba06ac11215b59788" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43983 } } peers { permanent_uuid: "a088e3d057b849d593f4d1972111e6c8" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 42453 } }
I20260811 19:02:52.355441 30521 leader_election.cc:288] T 9768ba0ee6164761b54ed6fdcd7865b2 P a088e3d057b849d593f4d1972111e6c8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fddff96ae4b84cdc86942277531be082 (127.13.109.2:34823), 7f911943ce2449aba06ac11215b59788 (127.13.109.3:43983)
W20260811 19:02:52.356448 30081 leader_election.cc:341] T 9768ba0ee6164761b54ed6fdcd7865b2 P a088e3d057b849d593f4d1972111e6c8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7f911943ce2449aba06ac11215b59788 (127.13.109.3:43983): Not found: Tablet not found: 9768ba0ee6164761b54ed6fdcd7865b2
W20260811 19:02:52.356557 30080 leader_election.cc:341] T 9768ba0ee6164761b54ed6fdcd7865b2 P a088e3d057b849d593f4d1972111e6c8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fddff96ae4b84cdc86942277531be082 (127.13.109.2:34823): Not found: Tablet not found: 9768ba0ee6164761b54ed6fdcd7865b2
I20260811 19:02:52.356623 30080 leader_election.cc:302] T 9768ba0ee6164761b54ed6fdcd7865b2 P a088e3d057b849d593f4d1972111e6c8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a088e3d057b849d593f4d1972111e6c8; no voters: 7f911943ce2449aba06ac11215b59788, fddff96ae4b84cdc86942277531be082
I20260811 19:02:52.356775 30521 raft_consensus.cc:2726] T 9768ba0ee6164761b54ed6fdcd7865b2 P a088e3d057b849d593f4d1972111e6c8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 19:02:52.383705 30522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:52.383249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:37073 (local address 127.13.109.3:48181)
0811 19:02:52.383541 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:52.383599 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:37073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":102,"threads_started":1}
I20260811 19:02:52.990906 30523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:52.990461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:37073 (local address 127.13.109.1:50851)
0811 19:02:52.990739 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:52.990804 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:37073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":84,"threads_started":1}
I20260811 19:02:53.176575 30524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:53.176101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:37073 (local address 127.13.109.2:37545)
0811 19:02:53.176393 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:53.176454 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:37073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":98,"threads_started":1}
I20260811 19:02:53.384886 30525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:53.384448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:37073 (local address 127.13.109.3:51439)
0811 19:02:53.384722 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:53.384785 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:37073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":84,"threads_started":1}
I20260811 19:02:53.992540 30526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:53.992008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:37073 (local address 127.13.109.1:35499)
0811 19:02:53.992364 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:53.992436 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:37073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":91,"threads_started":1}
I20260811 19:02:54.178112 30527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:54.177590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:37073 (local address 127.13.109.2:50851)
0811 19:02:54.177938 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:54.178005 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:37073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":91,"threads_started":1}
I20260811 19:02:54.374321 30528 raft_consensus.cc:482] T 9768ba0ee6164761b54ed6fdcd7865b2 P a088e3d057b849d593f4d1972111e6c8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:02:54.374481 30528 raft_consensus.cc:504] T 9768ba0ee6164761b54ed6fdcd7865b2 P a088e3d057b849d593f4d1972111e6c8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fddff96ae4b84cdc86942277531be082" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34823 } } peers { permanent_uuid: "7f911943ce2449aba06ac11215b59788" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43983 } } peers { permanent_uuid: "a088e3d057b849d593f4d1972111e6c8" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 42453 } }
I20260811 19:02:54.374948 30528 leader_election.cc:288] T 9768ba0ee6164761b54ed6fdcd7865b2 P a088e3d057b849d593f4d1972111e6c8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fddff96ae4b84cdc86942277531be082 (127.13.109.2:34823), 7f911943ce2449aba06ac11215b59788 (127.13.109.3:43983)
W20260811 19:02:54.375706 30081 leader_election.cc:341] T 9768ba0ee6164761b54ed6fdcd7865b2 P a088e3d057b849d593f4d1972111e6c8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7f911943ce2449aba06ac11215b59788 (127.13.109.3:43983): Not found: Tablet not found: 9768ba0ee6164761b54ed6fdcd7865b2
W20260811 19:02:54.375797 30080 leader_election.cc:341] T 9768ba0ee6164761b54ed6fdcd7865b2 P a088e3d057b849d593f4d1972111e6c8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fddff96ae4b84cdc86942277531be082 (127.13.109.2:34823): Not found: Tablet not found: 9768ba0ee6164761b54ed6fdcd7865b2
I20260811 19:02:54.375916 30080 leader_election.cc:302] T 9768ba0ee6164761b54ed6fdcd7865b2 P a088e3d057b849d593f4d1972111e6c8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a088e3d057b849d593f4d1972111e6c8; no voters: 7f911943ce2449aba06ac11215b59788, fddff96ae4b84cdc86942277531be082
I20260811 19:02:54.376068 30528 raft_consensus.cc:2726] T 9768ba0ee6164761b54ed6fdcd7865b2 P a088e3d057b849d593f4d1972111e6c8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 19:02:54.386308 30529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:54.385870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:37073 (local address 127.13.109.3:57069)
0811 19:02:54.386140 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:54.386206 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:37073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":79,"threads_started":1}
I20260811 19:02:54.994024 30530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:54.993501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:37073 (local address 127.13.109.1:35125)
0811 19:02:54.993852 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:54.993919 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:37073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":95,"threads_started":1}
I20260811 19:02:55.179577 30531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:55.179079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:37073 (local address 127.13.109.2:46359)
0811 19:02:55.179386 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:55.179466 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:37073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":86,"threads_started":1}
I20260811 19:02:55.387779 30532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:55.387270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:37073 (local address 127.13.109.3:42165)
0811 19:02:55.387579 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:55.387636 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:37073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":131,"threads_started":1}
I20260811 19:02:55.995491 30534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:55.995010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:37073 (local address 127.13.109.1:41203)
0811 19:02:55.995318 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:55.995385 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:37073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":85,"threads_started":1}
I20260811 19:02:56.181280 30535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:56.180736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:37073 (local address 127.13.109.2:47955)
0811 19:02:56.181096 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:56.181170 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:37073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":269,"spinlock_wait_cycles":2432,"thread_start_us":217,"threads_started":1}
I20260811 19:02:56.389353 30536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:56.388812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:37073 (local address 127.13.109.3:59963)
0811 19:02:56.389168 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:56.389244 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:37073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":88,"threads_started":1}
I20260811 19:02:56.997236 30537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:56.996517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:37073 (local address 127.13.109.1:49459)
0811 19:02:56.996831 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:56.997088 (+ 257us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:37073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":96,"threads_started":1}
I20260811 19:02:57.182821 30539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:57.182321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:37073 (local address 127.13.109.2:47521)
0811 19:02:57.182645 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:57.182713 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:37073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":100,"threads_started":1}
I20260811 19:02:57.390834 30540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:57.390352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:37073 (local address 127.13.109.3:53393)
0811 19:02:57.390655 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:57.390731 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:37073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":100,"threads_started":1}
I20260811 19:02:57.998734 30541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:57.998233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:37073 (local address 127.13.109.1:43391)
0811 19:02:57.998554 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:57.998624 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:37073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":98,"threads_started":1}
I20260811 19:02:58.184247 30542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:58.183760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:37073 (local address 127.13.109.2:49137)
0811 19:02:58.184068 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:58.184136 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:37073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":99,"threads_started":1}
I20260811 19:02:58.392338 30543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:58.391786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:37073 (local address 127.13.109.3:33273)
0811 19:02:58.392154 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:58.392230 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:37073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":100,"threads_started":1}
I20260811 19:02:58.445436 30544 raft_consensus.cc:482] T 9768ba0ee6164761b54ed6fdcd7865b2 P a088e3d057b849d593f4d1972111e6c8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:02:58.445603 30544 raft_consensus.cc:504] T 9768ba0ee6164761b54ed6fdcd7865b2 P a088e3d057b849d593f4d1972111e6c8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fddff96ae4b84cdc86942277531be082" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34823 } } peers { permanent_uuid: "7f911943ce2449aba06ac11215b59788" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43983 } } peers { permanent_uuid: "a088e3d057b849d593f4d1972111e6c8" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 42453 } }
I20260811 19:02:58.446257 30544 leader_election.cc:288] T 9768ba0ee6164761b54ed6fdcd7865b2 P a088e3d057b849d593f4d1972111e6c8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fddff96ae4b84cdc86942277531be082 (127.13.109.2:34823), 7f911943ce2449aba06ac11215b59788 (127.13.109.3:43983)
W20260811 19:02:58.447316 30080 leader_election.cc:341] T 9768ba0ee6164761b54ed6fdcd7865b2 P a088e3d057b849d593f4d1972111e6c8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fddff96ae4b84cdc86942277531be082 (127.13.109.2:34823): Not found: Tablet not found: 9768ba0ee6164761b54ed6fdcd7865b2
W20260811 19:02:58.447500 30081 leader_election.cc:341] T 9768ba0ee6164761b54ed6fdcd7865b2 P a088e3d057b849d593f4d1972111e6c8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7f911943ce2449aba06ac11215b59788 (127.13.109.3:43983): Not found: Tablet not found: 9768ba0ee6164761b54ed6fdcd7865b2
I20260811 19:02:58.447611 30081 leader_election.cc:302] T 9768ba0ee6164761b54ed6fdcd7865b2 P a088e3d057b849d593f4d1972111e6c8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a088e3d057b849d593f4d1972111e6c8; no voters: 7f911943ce2449aba06ac11215b59788, fddff96ae4b84cdc86942277531be082
I20260811 19:02:58.447860 30544 raft_consensus.cc:2726] T 9768ba0ee6164761b54ed6fdcd7865b2 P a088e3d057b849d593f4d1972111e6c8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 19:02:59.000026 30545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:58.999615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:37073 (local address 127.13.109.1:40637)
0811 19:02:58.999869 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:58.999927 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:37073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":74,"threads_started":1}
I20260811 19:02:59.185887 30546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:59.185357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:37073 (local address 127.13.109.2:34077)
0811 19:02:59.185644 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:59.185713 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:37073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":82,"threads_started":1}
I20260811 19:02:59.393849 30547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:59.393320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:37073 (local address 127.13.109.3:53357)
0811 19:02:59.393631 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:59.393710 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:37073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":99,"threads_started":1}
I20260811 19:03:00.001678 30548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:00.001049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:37073 (local address 127.13.109.1:42839)
0811 19:03:00.001477 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:00.001560 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:37073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":99,"threads_started":1}
I20260811 19:03:00.187304 30549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:00.186836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:37073 (local address 127.13.109.2:57155)
0811 19:03:00.187125 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:00.187194 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:37073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":81,"threads_started":1}
I20260811 19:03:00.395387 30550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:00.394944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:37073 (local address 127.13.109.3:33175)
0811 19:03:00.395218 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:00.395280 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:37073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":78,"threads_started":1}
I20260811 19:03:01.003480 30551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:01.002961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:37073 (local address 127.13.109.1:52529)
0811 19:03:01.003297 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:01.003375 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:37073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":103,"threads_started":1}
I20260811 19:03:01.188748 30552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:01.188287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:37073 (local address 127.13.109.2:53871)
0811 19:03:01.188581 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:01.188644 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:37073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":89,"threads_started":1}
I20260811 19:03:01.396785 30553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:01.396267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:37073 (local address 127.13.109.3:45207)
0811 19:03:01.396608 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:01.396681 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:37073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":153,"threads_started":1}
I20260811 19:03:01.869382 30555 raft_consensus.cc:482] T 9768ba0ee6164761b54ed6fdcd7865b2 P a088e3d057b849d593f4d1972111e6c8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:03:01.869536 30555 raft_consensus.cc:504] T 9768ba0ee6164761b54ed6fdcd7865b2 P a088e3d057b849d593f4d1972111e6c8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fddff96ae4b84cdc86942277531be082" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 34823 } } peers { permanent_uuid: "7f911943ce2449aba06ac11215b59788" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 43983 } } peers { permanent_uuid: "a088e3d057b849d593f4d1972111e6c8" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 42453 } }
I20260811 19:03:01.870126 30555 leader_election.cc:288] T 9768ba0ee6164761b54ed6fdcd7865b2 P a088e3d057b849d593f4d1972111e6c8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fddff96ae4b84cdc86942277531be082 (127.13.109.2:34823), 7f911943ce2449aba06ac11215b59788 (127.13.109.3:43983)
W20260811 19:03:01.871022 30081 leader_election.cc:341] T 9768ba0ee6164761b54ed6fdcd7865b2 P a088e3d057b849d593f4d1972111e6c8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7f911943ce2449aba06ac11215b59788 (127.13.109.3:43983): Not found: Tablet not found: 9768ba0ee6164761b54ed6fdcd7865b2
W20260811 19:03:01.871121 30080 leader_election.cc:341] T 9768ba0ee6164761b54ed6fdcd7865b2 P a088e3d057b849d593f4d1972111e6c8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fddff96ae4b84cdc86942277531be082 (127.13.109.2:34823): Not found: Tablet not found: 9768ba0ee6164761b54ed6fdcd7865b2
I20260811 19:03:01.871234 30080 leader_election.cc:302] T 9768ba0ee6164761b54ed6fdcd7865b2 P a088e3d057b849d593f4d1972111e6c8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a088e3d057b849d593f4d1972111e6c8; no voters: 7f911943ce2449aba06ac11215b59788, fddff96ae4b84cdc86942277531be082
I20260811 19:03:01.871402 30555 raft_consensus.cc:2726] T 9768ba0ee6164761b54ed6fdcd7865b2 P a088e3d057b849d593f4d1972111e6c8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 19:03:02.005061 30556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:02.004542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:37073 (local address 127.13.109.1:42323)
0811 19:03:02.004881 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:02.004948 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:37073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":122,"threads_started":1}
I20260811 19:03:02.190209 30557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:02.189680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:37073 (local address 127.13.109.2:49223)
0811 19:03:02.190020 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:02.190084 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:37073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":116,"threads_started":1}
I20260811 19:03:02.398211 30558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:02.397697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:37073 (local address 127.13.109.3:44771)
0811 19:03:02.398044 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:02.398108 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:37073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":127,"threads_started":1}
I20260811 19:03:03.006477 30559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:03.006025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:37073 (local address 127.13.109.1:52817)
0811 19:03:03.006306 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:03.006373 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:37073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":84,"threads_started":1}
I20260811 19:03:03.191588 30560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:03.191110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:37073 (local address 127.13.109.2:52113)
0811 19:03:03.191413 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:03.191479 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:37073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":108,"threads_started":1}
I20260811 19:03:03.399693 30561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:03.399094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:37073 (local address 127.13.109.3:47939)
0811 19:03:03.399526 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:03.399591 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:37073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":140,"threads_started":1}
I20260811 19:03:04.008183 30562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:04.007634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:37073 (local address 127.13.109.1:59433)
0811 19:03:04.007973 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:04.008047 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:37073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":100,"threads_started":1}
I20260811 19:03:04.193159 30563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:04.192631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:37073 (local address 127.13.109.2:54229)
0811 19:03:04.192942 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:04.193012 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:37073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":82,"threads_started":1}
I20260811 19:03:04.390220 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 30026
I20260811 19:03:04.401257 30565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:04.400790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:37073 (local address 127.13.109.3:35667)
0811 19:03:04.401084 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:04.401153 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:37073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":108,"threads_started":1}
I20260811 19:03:04.402496 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 30208
I20260811 19:03:04.413200 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 30353
2026-08-11T19:03:04Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19612 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:03:04 dist-test-slave-0kls krb5kdc[30569](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:03:04 dist-test-slave-0kls krb5kdc[30569](info): set up 2 sockets
Aug 11 19:03:04 dist-test-slave-0kls krb5kdc[30569](info): commencing operation
krb5kdc: starting...
W20260811 19:03:05.523765 13748 mini_kdc.cc:121] Time spent starting KDC: real 1.085s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/krb5kdc/test-user.keytab.
Aug 11 19:03:05 dist-test-slave-0kls krb5kdc[30569](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474985, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:03:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:03:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.109.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.109.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.109.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab.
Entry for principal kudu/127.13.109.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab.
I20260811 19:03:05.669960 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.62:36875
--webserver_interface=127.13.109.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab
--principal=kudu/127.13.109.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:32993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.109.62:36875
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:03:05.808002 30583 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:03:05.808377 30583 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:03:05.808465 30583 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:03:05.812776 30583 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 19:03:05.812906 30583 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 19:03:05.812932 30583 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 19:03:05.812956 30583 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:03:05.813000 30583 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:03:05.813073 30583 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:03:05.813143 30583 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:03:05.813181 30583 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:03:05.819113 30583 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:32993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.13.109.62:36875
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab
--principal=kudu/127.13.109.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.62:36875
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.109.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30583
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:03:05.820462 30583 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:03:05.821593 30583 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:03:05.830932 30592 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:03:05.831120 30583 server_base.cc:1034] running on GCE node
W20260811 19:03:05.830981 30589 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:03:05.830989 30590 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:03:05.831856 30583 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:03:05.833112 30583 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:03:05.834308 30583 hybrid_clock.cc:648] HybridClock initialized: now 1786474985834282 us; error 40 us; skew 500 ppm
Aug 11 19:03:05 dist-test-slave-0kls krb5kdc[30569](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474985, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:03:05.837090 30583 init.cc:377] Logged in from keytab as kudu/127.13.109.62@KRBTEST.COM (short username kudu)
I20260811 19:03:05.838191 30583 webserver.cc:460] Webserver started at http://127.13.109.62:42837/ using document root <none> and password file <none>
I20260811 19:03:05.838801 30583 fs_manager.cc:359] Metadata directory not provided
I20260811 19:03:05.838900 30583 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:03:05.839159 30583 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:03:05.841387 30583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/master-0/data/instance:
uuid: "7d3cc91d778246cdb59c93831c15d546"
format_stamp: "Formatted at 2026-08-11 19:03:05 on dist-test-slave-0kls"
server_key: "e47d5437894a1d90f80f2212a32b4840"
server_key_iv: "96c4a8f843d3b7c7b8f8538711c6ed69"
server_key_version: "encryptionkey@0"
I20260811 19:03:05.842043 30583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/master-0/wal/instance:
uuid: "7d3cc91d778246cdb59c93831c15d546"
format_stamp: "Formatted at 2026-08-11 19:03:05 on dist-test-slave-0kls"
server_key: "e47d5437894a1d90f80f2212a32b4840"
server_key_iv: "96c4a8f843d3b7c7b8f8538711c6ed69"
server_key_version: "encryptionkey@0"
I20260811 19:03:05.846225 30583 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 19:03:05.848886 30599 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:03:05.850328 30583 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 19:03:05.850488 30583 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/master-0/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/master-0/wal
uuid: "7d3cc91d778246cdb59c93831c15d546"
format_stamp: "Formatted at 2026-08-11 19:03:05 on dist-test-slave-0kls"
server_key: "e47d5437894a1d90f80f2212a32b4840"
server_key_iv: "96c4a8f843d3b7c7b8f8538711c6ed69"
server_key_version: "encryptionkey@0"
I20260811 19:03:05.850620 30583 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:03:05.866668 30583 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:03:05.869952 30583 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:03:05.870206 30583 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:03:05.877895 30583 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.62:36875
I20260811 19:03:05.878224 30655 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.62:36875 every 8 connection(s)
I20260811 19:03:05.878937 30583 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
I20260811 19:03:05.881914 30656 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:03:05.887938 30656 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d3cc91d778246cdb59c93831c15d546: Bootstrap starting.
I20260811 19:03:05.888598 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 30583
I20260811 19:03:05.888721 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/master-0/wal/instance
I20260811 19:03:05.889032 13748 external_mini_cluster.cc:1428] Setting key ce577e1da36037bad22508388901626a
I20260811 19:03:05.891319 30656 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7d3cc91d778246cdb59c93831c15d546: Neither blocks nor log segments found. Creating new log.
I20260811 19:03:05.892123 30656 log.cc:824] T 00000000000000000000000000000000 P 7d3cc91d778246cdb59c93831c15d546: Log is configured to *not* fsync() on all Append() calls
I20260811 19:03:05.894506 30656 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d3cc91d778246cdb59c93831c15d546: No bootstrap required, opened a new log
Aug 11 19:03:05 dist-test-slave-0kls krb5kdc[30569](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474985, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
I20260811 19:03:05.899329 30656 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7d3cc91d778246cdb59c93831c15d546 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d3cc91d778246cdb59c93831c15d546" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 36875 } }
I20260811 19:03:05.899602 30656 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7d3cc91d778246cdb59c93831c15d546 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:03:05.899693 30656 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7d3cc91d778246cdb59c93831c15d546 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d3cc91d778246cdb59c93831c15d546, State: Initialized, Role: FOLLOWER
I20260811 19:03:05.900235 30656 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7d3cc91d778246cdb59c93831c15d546 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d3cc91d778246cdb59c93831c15d546" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 36875 } }
I20260811 19:03:05.900446 30656 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7d3cc91d778246cdb59c93831c15d546 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:03:05.900545 30656 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7d3cc91d778246cdb59c93831c15d546 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:03:05.900667 30656 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7d3cc91d778246cdb59c93831c15d546 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:03:05.901718 30656 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7d3cc91d778246cdb59c93831c15d546 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d3cc91d778246cdb59c93831c15d546" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 36875 } }
I20260811 19:03:05.902108 30656 leader_election.cc:302] T 00000000000000000000000000000000 P 7d3cc91d778246cdb59c93831c15d546 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7d3cc91d778246cdb59c93831c15d546; no voters:
I20260811 19:03:05.902487 30656 leader_election.cc:288] T 00000000000000000000000000000000 P 7d3cc91d778246cdb59c93831c15d546 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:03:05.902594 30662 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7d3cc91d778246cdb59c93831c15d546 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:03:05.902832 30662 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7d3cc91d778246cdb59c93831c15d546 [term 1 LEADER]: Becoming Leader. State: Replica: 7d3cc91d778246cdb59c93831c15d546, State: Running, Role: LEADER
I20260811 19:03:05.903205 30662 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7d3cc91d778246cdb59c93831c15d546 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d3cc91d778246cdb59c93831c15d546" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 36875 } }
I20260811 19:03:05.903924 30656 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7d3cc91d778246cdb59c93831c15d546 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:03:05.904469 30662 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d3cc91d778246cdb59c93831c15d546 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7d3cc91d778246cdb59c93831c15d546. Latest consensus state: current_term: 1 leader_uuid: "7d3cc91d778246cdb59c93831c15d546" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d3cc91d778246cdb59c93831c15d546" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 36875 } } }
I20260811 19:03:05.905330 30662 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d3cc91d778246cdb59c93831c15d546 [sys.catalog]: This master's current role is: LEADER
I20260811 19:03:05.905203 30663 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d3cc91d778246cdb59c93831c15d546 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7d3cc91d778246cdb59c93831c15d546" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d3cc91d778246cdb59c93831c15d546" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 36875 } } }
I20260811 19:03:05.905674 30663 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d3cc91d778246cdb59c93831c15d546 [sys.catalog]: This master's current role is: LEADER
I20260811 19:03:05.905743 30660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:05.890711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50434 (local address 127.13.109.62:36875)
0811 19:03:05.891184 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:05.891201 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:05.891239 (+ 38us) server_negotiation.cc:407] Connection header received
0811 19:03:05.891988 (+ 749us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:05.892007 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:05.892422 (+ 415us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:05.892817 (+ 395us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:05.893690 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:05.894817 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:05.895986 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:05.896487 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:05.896614 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:05.899583 (+ 2969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:03:05.899589 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:03:05.899625 (+ 36us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:03:05.899661 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:03:05.901647 (+ 1986us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:05.902189 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:05.902196 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:05.902202 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:05.902383 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:05.902703 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:05.902706 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:05.902708 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:05.903250 (+ 542us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:03:05.904190 (+ 940us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:05.904371 (+ 181us) server_negotiation.cc:299] Negotiation successful
0811 19:03:05.904700 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":153,"threads_started":1}
I20260811 19:03:05.906407 30673 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:03:05.909986 30673 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:03:05.915629 30673 catalog_manager.cc:1347] Generated new cluster ID: aa1f3e5656714d4fb6a72e293346f4f4
I20260811 19:03:05.915753 30673 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:03:05.957998 30673 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:03:05.959213 30673 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:03:05.966594 30673 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7d3cc91d778246cdb59c93831c15d546: Generated new TSK 0
I20260811 19:03:05.967491 30673 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.109.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.109.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.109.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.1.keytab.
Entry for principal kudu/127.13.109.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.1.keytab.
I20260811 19:03:06.005059 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.1:0
--local_ip_for_outbound_sockets=127.13.109.1
--webserver_interface=127.13.109.1
--webserver_port=0
--tserver_master_addrs=127.13.109.62:36875
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.1.keytab
--principal=kudu/127.13.109.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:32993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:03:06.128304 30685 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:03:06.128585 30685 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:03:06.128679 30685 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:03:06.132903 30685 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:03:06.133029 30685 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:03:06.133126 30685 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.1
I20260811 19:03:06.138695 30685 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:32993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.1.keytab
--principal=kudu/127.13.109.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.109.1
--webserver_port=0
--tserver_master_addrs=127.13.109.62:36875
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30685
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.1
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:03:06.139961 30685 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:03:06.141077 30685 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:03:06.151567 30690 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:03:06.151607 30694 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:03:06.151623 30691 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:03:06.151918 30685 server_base.cc:1034] running on GCE node
I20260811 19:03:06.152400 30685 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:03:06.153172 30685 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:03:06.154379 30685 hybrid_clock.cc:648] HybridClock initialized: now 1786474986154340 us; error 49 us; skew 500 ppm
Aug 11 19:03:06 dist-test-slave-0kls krb5kdc[30569](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474986, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:03:06.157637 30685 init.cc:377] Logged in from keytab as kudu/127.13.109.1@KRBTEST.COM (short username kudu)
I20260811 19:03:06.158730 30685 webserver.cc:460] Webserver started at http://127.13.109.1:33679/ using document root <none> and password file <none>
I20260811 19:03:06.159319 30685 fs_manager.cc:359] Metadata directory not provided
I20260811 19:03:06.159384 30685 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:03:06.159577 30685 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:03:06.162038 30685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-0/data/instance:
uuid: "0b24c35b827f4b0badf0b66aec15cae9"
format_stamp: "Formatted at 2026-08-11 19:03:06 on dist-test-slave-0kls"
server_key: "1cd0cfa39f884ff13a23f2075b5f2bfa"
server_key_iv: "d92e0b1842bb9ef1b0a5661f58bebd7a"
server_key_version: "encryptionkey@0"
I20260811 19:03:06.162693 30685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-0/wal/instance:
uuid: "0b24c35b827f4b0badf0b66aec15cae9"
format_stamp: "Formatted at 2026-08-11 19:03:06 on dist-test-slave-0kls"
server_key: "1cd0cfa39f884ff13a23f2075b5f2bfa"
server_key_iv: "d92e0b1842bb9ef1b0a5661f58bebd7a"
server_key_version: "encryptionkey@0"
I20260811 19:03:06.167166 30685 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260811 19:03:06.170100 30701 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:03:06.171700 30685 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 19:03:06.171837 30685 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-0/wal
uuid: "0b24c35b827f4b0badf0b66aec15cae9"
format_stamp: "Formatted at 2026-08-11 19:03:06 on dist-test-slave-0kls"
server_key: "1cd0cfa39f884ff13a23f2075b5f2bfa"
server_key_iv: "d92e0b1842bb9ef1b0a5661f58bebd7a"
server_key_version: "encryptionkey@0"
I20260811 19:03:06.171957 30685 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:03:06.184563 30685 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:03:06.187836 30685 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:03:06.188014 30685 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:03:06.188611 30685 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:03:06.189464 30685 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:03:06.189513 30685 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:03:06.189555 30685 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:03:06.189571 30685 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:03:06.199807 30685 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.1:36839
I20260811 19:03:06.199828 30827 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.1:36839 every 8 connection(s)
I20260811 19:03:06.200837 30685 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
I20260811 19:03:06.204866 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 30685
I20260811 19:03:06.205014 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-0/wal/instance
I20260811 19:03:06.205389 13748 external_mini_cluster.cc:1428] Setting key 36fae589b5a265db1009d82d717501d0
Aug 11 19:03:06 dist-test-slave-0kls krb5kdc[30569](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474986, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.1@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.109.2@KRBTEST.COM; defaulting to no policy
I20260811 19:03:06.217338 30660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:06.202859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.1:56771 (local address 127.13.109.62:36875)
0811 19:03:06.203041 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:06.203046 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:06.204175 (+ 1129us) server_negotiation.cc:407] Connection header received
0811 19:03:06.205393 (+ 1218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:06.205399 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:06.205470 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:06.205572 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:06.207172 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:06.207965 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:06.209151 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:06.209394 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:06.209516 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:06.212740 (+ 3224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:03:06.212744 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:03:06.212746 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:03:06.212779 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:03:06.214579 (+ 1800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:06.215334 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:06.215338 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:06.215340 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:06.215442 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:06.215875 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:06.215881 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:06.215883 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:06.216119 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:03:06.216247 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:06.216907 (+ 660us) server_negotiation.cc:299] Negotiation successful
0811 19:03:06.217070 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 19:03:06.218326 30830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:06.203293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:36875 (local address 127.13.109.1:56771)
0811 19:03:06.204007 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:06.204061 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 19:03:06.205144 (+ 1083us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:03:06.205852 (+ 708us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:03:06.205864 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:03:06.206379 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:03:06.207012 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:03:06.207031 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:03:06.208107 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:03:06.208110 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:03:06.209006 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:03:06.209017 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:03:06.209582 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:03:06.209586 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:03:06.209766 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:06.210522 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:03:06.210540 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:03:06.212489 (+ 1949us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:03:06.214746 (+ 2257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:03:06.214755 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:03:06.214778 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:03:06.215188 (+ 410us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:03:06.215565 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:03:06.215568 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:03:06.215570 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:03:06.215760 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:03:06.216248 (+ 488us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:03:06.216258 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:03:06.216628 (+ 370us) client_negotiation.cc:769] Sending connection context
0811 19:03:06.216862 (+ 234us) client_negotiation.cc:240] Negotiation successful
0811 19:03:06.217269 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":122,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.109.2@KRBTEST.COM" created.
I20260811 19:03:06.219977 30828 heartbeater.cc:347] Connected to a master server at 127.13.109.62:36875
I20260811 19:03:06.220292 30828 heartbeater.cc:464] Registering TS with master...
I20260811 19:03:06.220853 30828 heartbeater.cc:511] Master 127.13.109.62:36875 requested a full tablet report, sending...
I20260811 19:03:06.222668 30617 ts_manager.cc:194] Registered new tserver with Master: 0b24c35b827f4b0badf0b66aec15cae9 (127.13.109.1:36839)
I20260811 19:03:06.224036 30617 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.109.1@KRBTEST.COM'} at 127.13.109.1:56771
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.109.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.2.keytab.
Entry for principal kudu/127.13.109.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.2.keytab.
I20260811 19:03:06.240458 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.2:0
--local_ip_for_outbound_sockets=127.13.109.2
--webserver_interface=127.13.109.2
--webserver_port=0
--tserver_master_addrs=127.13.109.62:36875
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.2.keytab
--principal=kudu/127.13.109.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:32993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:03:06.359314 30833 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:03:06.359634 30833 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:03:06.359759 30833 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:03:06.363970 30833 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:03:06.364063 30833 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:03:06.364153 30833 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.2
I20260811 19:03:06.369686 30833 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:32993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.2.keytab
--principal=kudu/127.13.109.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.109.2
--webserver_port=0
--tserver_master_addrs=127.13.109.62:36875
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30833
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.2
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:03:06.370983 30833 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:03:06.372133 30833 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 19:03:06.382665 30833 server_base.cc:1034] running on GCE node
W20260811 19:03:06.382578 30839 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:03:06.382604 30842 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:03:06.382573 30838 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:03:06.383193 30833 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:03:06.383837 30833 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:03:06.385028 30833 hybrid_clock.cc:648] HybridClock initialized: now 1786474986384987 us; error 52 us; skew 500 ppm
Aug 11 19:03:06 dist-test-slave-0kls krb5kdc[30569](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474986, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:03:06.387913 30833 init.cc:377] Logged in from keytab as kudu/127.13.109.2@KRBTEST.COM (short username kudu)
I20260811 19:03:06.388881 30833 webserver.cc:460] Webserver started at http://127.13.109.2:36677/ using document root <none> and password file <none>
I20260811 19:03:06.389473 30833 fs_manager.cc:359] Metadata directory not provided
I20260811 19:03:06.389583 30833 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:03:06.389889 30833 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:03:06.391981 30833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-1/data/instance:
uuid: "e1052c249dde44e6901a8b7840c6e1bb"
format_stamp: "Formatted at 2026-08-11 19:03:06 on dist-test-slave-0kls"
server_key: "6383c25d81e363f2d5dab237d58d0b1d"
server_key_iv: "9d9dc05dcc7917ea22074ebcd087da87"
server_key_version: "encryptionkey@0"
I20260811 19:03:06.392561 30833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-1/wal/instance:
uuid: "e1052c249dde44e6901a8b7840c6e1bb"
format_stamp: "Formatted at 2026-08-11 19:03:06 on dist-test-slave-0kls"
server_key: "6383c25d81e363f2d5dab237d58d0b1d"
server_key_iv: "9d9dc05dcc7917ea22074ebcd087da87"
server_key_version: "encryptionkey@0"
I20260811 19:03:06.396551 30833 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260811 19:03:06.399003 30850 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:03:06.400202 30833 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 19:03:06.400372 30833 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-1/wal
uuid: "e1052c249dde44e6901a8b7840c6e1bb"
format_stamp: "Formatted at 2026-08-11 19:03:06 on dist-test-slave-0kls"
server_key: "6383c25d81e363f2d5dab237d58d0b1d"
server_key_iv: "9d9dc05dcc7917ea22074ebcd087da87"
server_key_version: "encryptionkey@0"
I20260811 19:03:06.400487 30833 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:03:06.415354 30833 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:03:06.418322 30833 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:03:06.418543 30833 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:03:06.419173 30833 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:03:06.420082 30833 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:03:06.420166 30833 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:03:06.420260 30833 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:03:06.420312 30833 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:03:06.429623 30833 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.2:45197
I20260811 19:03:06.429647 30980 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.2:45197 every 8 connection(s)
I20260811 19:03:06.430676 30833 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
I20260811 19:03:06.438151 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 30833
I20260811 19:03:06.438257 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-1/wal/instance
I20260811 19:03:06.438565 13748 external_mini_cluster.cc:1428] Setting key 49a9e877abc949d8fff0981dffa72137
Aug 11 19:03:06 dist-test-slave-0kls krb5kdc[30569](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474986, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.2@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
I20260811 19:03:06.444841 30660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:06.432769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.2:50245 (local address 127.13.109.62:36875)
0811 19:03:06.432932 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:06.432937 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:06.434215 (+ 1278us) server_negotiation.cc:407] Connection header received
0811 19:03:06.435090 (+ 875us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:06.435093 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:06.435140 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:06.435219 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:06.436377 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:06.436903 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:06.438099 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:06.438435 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:06.438560 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:06.441117 (+ 2557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:03:06.441126 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:03:06.441128 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:03:06.441152 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:03:06.442671 (+ 1519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:06.443210 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:06.443213 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:06.443215 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:06.443316 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:06.443670 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:06.443675 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:06.443677 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:06.443877 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:03:06.443991 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:06.444503 (+ 512us) server_negotiation.cc:299] Negotiation successful
0811 19:03:06.444639 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 19:03:06.445676 30983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:06.433246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:36875 (local address 127.13.109.2:50245)
0811 19:03:06.434062 (+ 816us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:06.434105 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 19:03:06.434890 (+ 785us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:03:06.435394 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:03:06.435400 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:03:06.435755 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:03:06.436249 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:03:06.436263 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:03:06.437011 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:03:06.437013 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:03:06.437969 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:03:06.437980 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:03:06.438603 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:03:06.438607 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:03:06.438804 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:06.439402 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:03:06.439419 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:03:06.440945 (+ 1526us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:03:06.442807 (+ 1862us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:03:06.442813 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:03:06.442828 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:03:06.443092 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:03:06.443413 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:03:06.443417 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:03:06.443419 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:03:06.443567 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:03:06.443991 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:03:06.443999 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:03:06.444261 (+ 262us) client_negotiation.cc:769] Sending connection context
0811 19:03:06.444458 (+ 197us) client_negotiation.cc:240] Negotiation successful
0811 19:03:06.444797 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":619,"thread_start_us":114,"threads_started":1}
I20260811 19:03:06.446911 30981 heartbeater.cc:347] Connected to a master server at 127.13.109.62:36875
I20260811 19:03:06.447181 30981 heartbeater.cc:464] Registering TS with master...
I20260811 19:03:06.447693 30981 heartbeater.cc:511] Master 127.13.109.62:36875 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.13.109.3@KRBTEST.COM; defaulting to no policy
I20260811 19:03:06.448848 30617 ts_manager.cc:194] Registered new tserver with Master: e1052c249dde44e6901a8b7840c6e1bb (127.13.109.2:45197)
I20260811 19:03:06.449455 30617 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.109.2@KRBTEST.COM'} at 127.13.109.2:50245
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.109.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.109.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.3.keytab.
Entry for principal kudu/127.13.109.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.3.keytab.
I20260811 19:03:06.468751 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.3:0
--local_ip_for_outbound_sockets=127.13.109.3
--webserver_interface=127.13.109.3
--webserver_port=0
--tserver_master_addrs=127.13.109.62:36875
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.3.keytab
--principal=kudu/127.13.109.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:32993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:03:06.586897 30986 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:03:06.587177 30986 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:03:06.587224 30986 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:03:06.591311 30986 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:03:06.591384 30986 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:03:06.591444 30986 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.3
I20260811 19:03:06.597083 30986 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:32993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.3.keytab
--principal=kudu/127.13.109.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.109.3
--webserver_port=0
--tserver_master_addrs=127.13.109.62:36875
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30986
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.3
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:03:06.598305 30986 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:03:06.599323 30986 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:03:06.609987 30993 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:03:06.610018 30986 server_base.cc:1034] running on GCE node
W20260811 19:03:06.609990 30996 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:03:06.610333 30992 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:03:06.610781 30986 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:03:06.611491 30986 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:03:06.612720 30986 hybrid_clock.cc:648] HybridClock initialized: now 1786474986612706 us; error 35 us; skew 500 ppm
Aug 11 19:03:06 dist-test-slave-0kls krb5kdc[30569](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474986, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:03:06.615744 30986 init.cc:377] Logged in from keytab as kudu/127.13.109.3@KRBTEST.COM (short username kudu)
I20260811 19:03:06.616878 30986 webserver.cc:460] Webserver started at http://127.13.109.3:37157/ using document root <none> and password file <none>
I20260811 19:03:06.617518 30986 fs_manager.cc:359] Metadata directory not provided
I20260811 19:03:06.617619 30986 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:03:06.617937 30986 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:03:06.620260 30986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-2/data/instance:
uuid: "a2336f52031842798c0dabb12b7e06e6"
format_stamp: "Formatted at 2026-08-11 19:03:06 on dist-test-slave-0kls"
server_key: "7a614070ce0fa326c55a07c0c28a6cf1"
server_key_iv: "f0757db09afeab1fd0cf5f0af21d1f3e"
server_key_version: "encryptionkey@0"
I20260811 19:03:06.620927 30986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-2/wal/instance:
uuid: "a2336f52031842798c0dabb12b7e06e6"
format_stamp: "Formatted at 2026-08-11 19:03:06 on dist-test-slave-0kls"
server_key: "7a614070ce0fa326c55a07c0c28a6cf1"
server_key_iv: "f0757db09afeab1fd0cf5f0af21d1f3e"
server_key_version: "encryptionkey@0"
I20260811 19:03:06.625506 30986 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 19:03:06.628760 31003 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:03:06.630339 30986 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 19:03:06.630513 30986 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-2/wal
uuid: "a2336f52031842798c0dabb12b7e06e6"
format_stamp: "Formatted at 2026-08-11 19:03:06 on dist-test-slave-0kls"
server_key: "7a614070ce0fa326c55a07c0c28a6cf1"
server_key_iv: "f0757db09afeab1fd0cf5f0af21d1f3e"
server_key_version: "encryptionkey@0"
I20260811 19:03:06.630661 30986 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:03:06.663316 30986 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:03:06.667258 30986 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:03:06.667506 30986 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:03:06.668239 30986 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:03:06.669274 30986 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:03:06.669358 30986 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:03:06.669440 30986 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:03:06.669488 30986 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:03:06.680332 30986 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.3:40165
I20260811 19:03:06.680356 31130 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.3:40165 every 8 connection(s)
I20260811 19:03:06.681409 30986 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
I20260811 19:03:06.687479 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 30986
I20260811 19:03:06.687593 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-2/wal/instance
I20260811 19:03:06.687919 13748 external_mini_cluster.cc:1428] Setting key 504b6a5ae425890cef702deae8a046db
Aug 11 19:03:06 dist-test-slave-0kls krb5kdc[30569](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474986, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.3@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
I20260811 19:03:06.696717 30660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:06.683590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.3:49275 (local address 127.13.109.62:36875)
0811 19:03:06.683752 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:06.683756 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:06.684687 (+ 931us) server_negotiation.cc:407] Connection header received
0811 19:03:06.686195 (+ 1508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:06.686199 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:06.686267 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:06.686385 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:06.687780 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:06.688398 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:06.689910 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:06.690295 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:06.690422 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:06.693093 (+ 2671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:03:06.693097 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:03:06.693099 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:03:06.693129 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:03:06.694629 (+ 1500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:06.695204 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:06.695207 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:06.695208 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:06.695293 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:06.695622 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:06.695624 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:06.695625 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:06.695830 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:03:06.695909 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:06.696453 (+ 544us) server_negotiation.cc:299] Negotiation successful
0811 19:03:06.696577 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 19:03:06.697739 31133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:06.684015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:36875 (local address 127.13.109.3:49275)
0811 19:03:06.684507 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:06.684559 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 19:03:06.685946 (+ 1387us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:03:06.686587 (+ 641us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:03:06.686596 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:03:06.687002 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:03:06.687591 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:03:06.687613 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:03:06.688568 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:03:06.688571 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:03:06.689731 (+ 1160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:03:06.689743 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:03:06.690472 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:03:06.690476 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:03:06.690660 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:06.691229 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:03:06.691247 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:03:06.692881 (+ 1634us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:03:06.694761 (+ 1880us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:03:06.694768 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:03:06.694788 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:03:06.695107 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:03:06.695378 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:03:06.695380 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:03:06.695381 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:03:06.695540 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:03:06.695925 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:03:06.695931 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:03:06.696225 (+ 294us) client_negotiation.cc:769] Sending connection context
0811 19:03:06.696403 (+ 178us) client_negotiation.cc:240] Negotiation successful
0811 19:03:06.696718 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":117,"threads_started":1}
I20260811 19:03:06.699153 31131 heartbeater.cc:347] Connected to a master server at 127.13.109.62:36875
I20260811 19:03:06.699424 31131 heartbeater.cc:464] Registering TS with master...
I20260811 19:03:06.699988 31131 heartbeater.cc:511] Master 127.13.109.62:36875 requested a full tablet report, sending...
I20260811 19:03:06.701220 30617 ts_manager.cc:194] Registered new tserver with Master: a2336f52031842798c0dabb12b7e06e6 (127.13.109.3:40165)
I20260811 19:03:06.701993 30617 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.109.3@KRBTEST.COM'} at 127.13.109.3:49275
I20260811 19:03:06.702545 13748 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 19:03:06.713326 30660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:06.704885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50436 (local address 127.13.109.62:36875)
0811 19:03:06.705035 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:06.705040 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:06.705321 (+ 281us) server_negotiation.cc:407] Connection header received
0811 19:03:06.705468 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:06.705471 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:06.705527 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:06.705618 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:06.706464 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:06.707004 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:06.707882 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:06.708123 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:06.708235 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:06.709729 (+ 1494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:03:06.709733 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:03:06.709736 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:03:06.709773 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:03:06.711342 (+ 1569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:06.711845 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:06.711848 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:06.711850 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:06.711955 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:06.712285 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:06.712289 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:06.712290 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:06.712578 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:03:06.712727 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:06.713003 (+ 276us) server_negotiation.cc:299] Negotiation successful
0811 19:03:06.713137 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 19:03:06.717624 30616 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50436:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 19:03:06.721275 30616 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 19:03:06.738482 31143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:06.731239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.2:45197 (local address 127.0.0.1:55102)
0811 19:03:06.731778 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:06.731833 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 19:03:06.731970 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:03:06.733163 (+ 1193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:03:06.733174 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:03:06.733207 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:03:06.733503 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:03:06.733510 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:03:06.735255 (+ 1745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:03:06.735259 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:03:06.736906 (+ 1647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:03:06.736919 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:03:06.737823 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:03:06.737827 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:03:06.737957 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:06.737999 (+ 42us) client_negotiation.cc:769] Sending connection context
0811 19:03:06.738130 (+ 131us) client_negotiation.cc:240] Negotiation successful
0811 19:03:06.738296 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":363,"mutex_wait_us":213,"thread_start_us":95,"threads_started":1}
I20260811 19:03:06.738620 31144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:06.731418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55102 (local address 127.13.109.2:45197)
0811 19:03:06.732580 (+ 1162us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:06.732595 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:06.732620 (+ 25us) server_negotiation.cc:407] Connection header received
0811 19:03:06.732704 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:06.732711 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:06.732938 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:06.733108 (+ 170us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:03:06.733643 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:06.735105 (+ 1462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:06.737066 (+ 1961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:06.737653 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:06.737798 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:06.738013 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:06.738166 (+ 153us) server_negotiation.cc:299] Negotiation successful
0811 19:03:06.738457 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1048,"thread_start_us":152,"threads_started":1}
I20260811 19:03:06.738816 31146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:06.732010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58208 (local address 127.13.109.1:36839)
0811 19:03:06.732788 (+ 778us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:06.732797 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:06.732824 (+ 27us) server_negotiation.cc:407] Connection header received
0811 19:03:06.732908 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:06.732912 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:06.733103 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:06.733667 (+ 564us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:03:06.733846 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:06.735090 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:06.736788 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:06.737702 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:06.738280 (+ 578us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:06.738433 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:06.738488 (+ 55us) server_negotiation.cc:299] Negotiation successful
0811 19:03:06.738656 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":667,"thread_start_us":83,"threads_started":1}
I20260811 19:03:06.738919 31147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:06.732003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.3:40165 (local address 127.0.0.1:47822)
0811 19:03:06.732383 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:06.732399 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 19:03:06.732507 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:03:06.733388 (+ 881us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:03:06.733391 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:03:06.733412 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:03:06.733547 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:03:06.733557 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:03:06.735115 (+ 1558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:03:06.735120 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:03:06.736906 (+ 1786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:03:06.736919 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:03:06.738322 (+ 1403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:03:06.738326 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:03:06.738431 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:06.738461 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 19:03:06.738542 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 19:03:06.738635 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"mutex_wait_us":365,"thread_start_us":109,"threads_started":1}
I20260811 19:03:06.739382 31148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:06.732465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47822 (local address 127.13.109.3:40165)
0811 19:03:06.732949 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:06.732962 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:06.732984 (+ 22us) server_negotiation.cc:407] Connection header received
0811 19:03:06.733062 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:06.733067 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:06.733235 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:06.733414 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:03:06.733694 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:06.734947 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:06.737052 (+ 2105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:06.737971 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:06.738079 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:06.738254 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:06.739114 (+ 860us) server_negotiation.cc:299] Negotiation successful
0811 19:03:06.739248 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":182,"threads_started":1}
I20260811 19:03:06.742120 31145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:06.731427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.1:36839 (local address 127.0.0.1:58208)
0811 19:03:06.732238 (+ 811us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:06.732252 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 19:03:06.732332 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:03:06.733265 (+ 933us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:03:06.733268 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:03:06.733283 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:03:06.733491 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:03:06.733500 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:03:06.735255 (+ 1755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:03:06.735259 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:03:06.736629 (+ 1370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:03:06.736641 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:03:06.737901 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:03:06.737905 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:03:06.738016 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:06.738060 (+ 44us) client_negotiation.cc:769] Sending connection context
0811 19:03:06.738130 (+ 70us) client_negotiation.cc:240] Negotiation successful
0811 19:03:06.741961 (+ 3831us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":735,"mutex_wait_us":3919,"thread_start_us":153,"threads_started":1}
I20260811 19:03:06.742153 31058 tablet_service.cc:1467] Processing CreateTablet for tablet e669b630ee114961ad830647ab90abc3 (DEFAULT_TABLE table=test-table [id=ebc7efa4b6cb46b2ae51391c64a0d047]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:03:06.742156 30905 tablet_service.cc:1467] Processing CreateTablet for tablet e669b630ee114961ad830647ab90abc3 (DEFAULT_TABLE table=test-table [id=ebc7efa4b6cb46b2ae51391c64a0d047]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:03:06.743469 30905 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e669b630ee114961ad830647ab90abc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:03:06.743484 31058 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e669b630ee114961ad830647ab90abc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:03:06.745657 30757 tablet_service.cc:1467] Processing CreateTablet for tablet e669b630ee114961ad830647ab90abc3 (DEFAULT_TABLE table=test-table [id=ebc7efa4b6cb46b2ae51391c64a0d047]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:03:06.746963 30757 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e669b630ee114961ad830647ab90abc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:03:06.750560 31149 tablet_bootstrap.cc:492] T e669b630ee114961ad830647ab90abc3 P e1052c249dde44e6901a8b7840c6e1bb: Bootstrap starting.
I20260811 19:03:06.752341 31150 tablet_bootstrap.cc:492] T e669b630ee114961ad830647ab90abc3 P a2336f52031842798c0dabb12b7e06e6: Bootstrap starting.
I20260811 19:03:06.753921 31149 tablet_bootstrap.cc:654] T e669b630ee114961ad830647ab90abc3 P e1052c249dde44e6901a8b7840c6e1bb: Neither blocks nor log segments found. Creating new log.
I20260811 19:03:06.754823 31149 log.cc:824] T e669b630ee114961ad830647ab90abc3 P e1052c249dde44e6901a8b7840c6e1bb: Log is configured to *not* fsync() on all Append() calls
I20260811 19:03:06.755432 31150 tablet_bootstrap.cc:654] T e669b630ee114961ad830647ab90abc3 P a2336f52031842798c0dabb12b7e06e6: Neither blocks nor log segments found. Creating new log.
I20260811 19:03:06.755661 31152 tablet_bootstrap.cc:492] T e669b630ee114961ad830647ab90abc3 P 0b24c35b827f4b0badf0b66aec15cae9: Bootstrap starting.
I20260811 19:03:06.756426 31150 log.cc:824] T e669b630ee114961ad830647ab90abc3 P a2336f52031842798c0dabb12b7e06e6: Log is configured to *not* fsync() on all Append() calls
I20260811 19:03:06.757264 31149 tablet_bootstrap.cc:492] T e669b630ee114961ad830647ab90abc3 P e1052c249dde44e6901a8b7840c6e1bb: No bootstrap required, opened a new log
I20260811 19:03:06.757488 31149 ts_tablet_manager.cc:1397] T e669b630ee114961ad830647ab90abc3 P e1052c249dde44e6901a8b7840c6e1bb: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260811 19:03:06.758759 31150 tablet_bootstrap.cc:492] T e669b630ee114961ad830647ab90abc3 P a2336f52031842798c0dabb12b7e06e6: No bootstrap required, opened a new log
I20260811 19:03:06.758986 31150 ts_tablet_manager.cc:1397] T e669b630ee114961ad830647ab90abc3 P a2336f52031842798c0dabb12b7e06e6: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 19:03:06.760293 31152 tablet_bootstrap.cc:654] T e669b630ee114961ad830647ab90abc3 P 0b24c35b827f4b0badf0b66aec15cae9: Neither blocks nor log segments found. Creating new log.
I20260811 19:03:06.761093 31149 raft_consensus.cc:348] T e669b630ee114961ad830647ab90abc3 P e1052c249dde44e6901a8b7840c6e1bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1052c249dde44e6901a8b7840c6e1bb" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 45197 } } peers { permanent_uuid: "0b24c35b827f4b0badf0b66aec15cae9" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 36839 } } peers { permanent_uuid: "a2336f52031842798c0dabb12b7e06e6" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 40165 } }
I20260811 19:03:06.761320 31152 log.cc:824] T e669b630ee114961ad830647ab90abc3 P 0b24c35b827f4b0badf0b66aec15cae9: Log is configured to *not* fsync() on all Append() calls
I20260811 19:03:06.761395 31149 raft_consensus.cc:374] T e669b630ee114961ad830647ab90abc3 P e1052c249dde44e6901a8b7840c6e1bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:03:06.761451 31149 raft_consensus.cc:729] T e669b630ee114961ad830647ab90abc3 P e1052c249dde44e6901a8b7840c6e1bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1052c249dde44e6901a8b7840c6e1bb, State: Initialized, Role: FOLLOWER
I20260811 19:03:06.761982 31149 consensus_queue.cc:260] T e669b630ee114961ad830647ab90abc3 P e1052c249dde44e6901a8b7840c6e1bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1052c249dde44e6901a8b7840c6e1bb" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 45197 } } peers { permanent_uuid: "0b24c35b827f4b0badf0b66aec15cae9" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 36839 } } peers { permanent_uuid: "a2336f52031842798c0dabb12b7e06e6" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 40165 } }
I20260811 19:03:06.762781 30981 heartbeater.cc:503] Master 127.13.109.62:36875 was elected leader, sending a full tablet report...
I20260811 19:03:06.763054 31149 ts_tablet_manager.cc:1428] T e669b630ee114961ad830647ab90abc3 P e1052c249dde44e6901a8b7840c6e1bb: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260811 19:03:06.763367 31150 raft_consensus.cc:348] T e669b630ee114961ad830647ab90abc3 P a2336f52031842798c0dabb12b7e06e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1052c249dde44e6901a8b7840c6e1bb" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 45197 } } peers { permanent_uuid: "0b24c35b827f4b0badf0b66aec15cae9" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 36839 } } peers { permanent_uuid: "a2336f52031842798c0dabb12b7e06e6" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 40165 } }
I20260811 19:03:06.763689 31150 raft_consensus.cc:374] T e669b630ee114961ad830647ab90abc3 P a2336f52031842798c0dabb12b7e06e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:03:06.763762 31150 raft_consensus.cc:729] T e669b630ee114961ad830647ab90abc3 P a2336f52031842798c0dabb12b7e06e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2336f52031842798c0dabb12b7e06e6, State: Initialized, Role: FOLLOWER
I20260811 19:03:06.764245 31150 consensus_queue.cc:260] T e669b630ee114961ad830647ab90abc3 P a2336f52031842798c0dabb12b7e06e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1052c249dde44e6901a8b7840c6e1bb" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 45197 } } peers { permanent_uuid: "0b24c35b827f4b0badf0b66aec15cae9" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 36839 } } peers { permanent_uuid: "a2336f52031842798c0dabb12b7e06e6" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 40165 } }
I20260811 19:03:06.765010 31131 heartbeater.cc:503] Master 127.13.109.62:36875 was elected leader, sending a full tablet report...
I20260811 19:03:06.765368 31150 ts_tablet_manager.cc:1428] T e669b630ee114961ad830647ab90abc3 P a2336f52031842798c0dabb12b7e06e6: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 19:03:06.766464 31152 tablet_bootstrap.cc:492] T e669b630ee114961ad830647ab90abc3 P 0b24c35b827f4b0badf0b66aec15cae9: No bootstrap required, opened a new log
I20260811 19:03:06.766764 31152 ts_tablet_manager.cc:1397] T e669b630ee114961ad830647ab90abc3 P 0b24c35b827f4b0badf0b66aec15cae9: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.000s
I20260811 19:03:06.771587 31152 raft_consensus.cc:348] T e669b630ee114961ad830647ab90abc3 P 0b24c35b827f4b0badf0b66aec15cae9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1052c249dde44e6901a8b7840c6e1bb" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 45197 } } peers { permanent_uuid: "0b24c35b827f4b0badf0b66aec15cae9" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 36839 } } peers { permanent_uuid: "a2336f52031842798c0dabb12b7e06e6" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 40165 } }
I20260811 19:03:06.771911 31152 raft_consensus.cc:374] T e669b630ee114961ad830647ab90abc3 P 0b24c35b827f4b0badf0b66aec15cae9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:03:06.771950 31152 raft_consensus.cc:729] T e669b630ee114961ad830647ab90abc3 P 0b24c35b827f4b0badf0b66aec15cae9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b24c35b827f4b0badf0b66aec15cae9, State: Initialized, Role: FOLLOWER
I20260811 19:03:06.772384 31152 consensus_queue.cc:260] T e669b630ee114961ad830647ab90abc3 P 0b24c35b827f4b0badf0b66aec15cae9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1052c249dde44e6901a8b7840c6e1bb" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 45197 } } peers { permanent_uuid: "0b24c35b827f4b0badf0b66aec15cae9" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 36839 } } peers { permanent_uuid: "a2336f52031842798c0dabb12b7e06e6" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 40165 } }
I20260811 19:03:06.773197 30828 heartbeater.cc:503] Master 127.13.109.62:36875 was elected leader, sending a full tablet report...
I20260811 19:03:06.773597 31152 ts_tablet_manager.cc:1428] T e669b630ee114961ad830647ab90abc3 P 0b24c35b827f4b0badf0b66aec15cae9: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
W20260811 19:03:06.931932 30982 tablet.cc:2369] T e669b630ee114961ad830647ab90abc3 P e1052c249dde44e6901a8b7840c6e1bb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 19:03:06.932521 31132 tablet.cc:2369] T e669b630ee114961ad830647ab90abc3 P a2336f52031842798c0dabb12b7e06e6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 19:03:06.952323 30829 tablet.cc:2369] T e669b630ee114961ad830647ab90abc3 P 0b24c35b827f4b0badf0b66aec15cae9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 19:03:07.170198 31156 raft_consensus.cc:482] T e669b630ee114961ad830647ab90abc3 P e1052c249dde44e6901a8b7840c6e1bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:03:07.170504 31156 raft_consensus.cc:504] T e669b630ee114961ad830647ab90abc3 P e1052c249dde44e6901a8b7840c6e1bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1052c249dde44e6901a8b7840c6e1bb" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 45197 } } peers { permanent_uuid: "0b24c35b827f4b0badf0b66aec15cae9" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 36839 } } peers { permanent_uuid: "a2336f52031842798c0dabb12b7e06e6" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 40165 } }
I20260811 19:03:07.171757 31156 leader_election.cc:288] T e669b630ee114961ad830647ab90abc3 P e1052c249dde44e6901a8b7840c6e1bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0b24c35b827f4b0badf0b66aec15cae9 (127.13.109.1:36839), a2336f52031842798c0dabb12b7e06e6 (127.13.109.3:40165)
I20260811 19:03:07.176445 31146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:07.172338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.2:52983 (local address 127.13.109.1:36839)
0811 19:03:07.172533 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:07.172539 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:07.172657 (+ 118us) server_negotiation.cc:407] Connection header received
0811 19:03:07.172852 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:07.172855 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:07.172910 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:07.172997 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:03:07.173370 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:07.174024 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:07.175247 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:07.175802 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:07.175901 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:07.175946 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:07.176193 (+ 247us) server_negotiation.cc:299] Negotiation successful
0811 19:03:07.176282 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260811 19:03:07.176462 31163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:07.172147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.1:36839 (local address 127.13.109.2:52983)
0811 19:03:07.172531 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:07.172550 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 19:03:07.172719 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:03:07.173011 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:03:07.173014 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:03:07.173037 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:03:07.173215 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:03:07.173222 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:03:07.174141 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:03:07.174144 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:03:07.175102 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:03:07.175114 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:03:07.175925 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:03:07.175929 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:03:07.176048 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:07.176083 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 19:03:07.176159 (+ 76us) client_negotiation.cc:240] Negotiation successful
0811 19:03:07.176255 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"mutex_wait_us":1,"thread_start_us":136,"threads_started":1}
I20260811 19:03:07.176584 31148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:07.172775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.2:55159 (local address 127.13.109.3:40165)
0811 19:03:07.172912 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:07.172916 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:07.173123 (+ 207us) server_negotiation.cc:407] Connection header received
0811 19:03:07.173228 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:07.173231 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:07.173289 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:07.173368 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:03:07.173664 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:07.174332 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:07.175412 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:07.175981 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:07.176114 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:07.176179 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:07.176372 (+ 193us) server_negotiation.cc:299] Negotiation successful
0811 19:03:07.176452 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 19:03:07.176868 31164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:07.172621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.3:40165 (local address 127.13.109.2:55159)
0811 19:03:07.172963 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:07.172980 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 19:03:07.173126 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:03:07.173400 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:03:07.173402 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:03:07.173420 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:03:07.173544 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:03:07.173551 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:03:07.174448 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:03:07.174451 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:03:07.175277 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:03:07.175287 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:03:07.176105 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:03:07.176108 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:03:07.176231 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:07.176254 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 19:03:07.176332 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 19:03:07.176404 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"mutex_wait_us":1,"thread_start_us":83,"threads_started":1}
I20260811 19:03:07.177405 30777 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e669b630ee114961ad830647ab90abc3" candidate_uuid: "e1052c249dde44e6901a8b7840c6e1bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b24c35b827f4b0badf0b66aec15cae9" is_pre_election: true
I20260811 19:03:07.177663 31080 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e669b630ee114961ad830647ab90abc3" candidate_uuid: "e1052c249dde44e6901a8b7840c6e1bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2336f52031842798c0dabb12b7e06e6" is_pre_election: true
I20260811 19:03:07.177847 30777 raft_consensus.cc:2454] T e669b630ee114961ad830647ab90abc3 P 0b24c35b827f4b0badf0b66aec15cae9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1052c249dde44e6901a8b7840c6e1bb in term 0.
I20260811 19:03:07.177976 31080 raft_consensus.cc:2454] T e669b630ee114961ad830647ab90abc3 P a2336f52031842798c0dabb12b7e06e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1052c249dde44e6901a8b7840c6e1bb in term 0.
I20260811 19:03:07.178416 30851 leader_election.cc:302] T e669b630ee114961ad830647ab90abc3 P e1052c249dde44e6901a8b7840c6e1bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b24c35b827f4b0badf0b66aec15cae9, e1052c249dde44e6901a8b7840c6e1bb; no voters:
I20260811 19:03:07.178701 31156 raft_consensus.cc:2781] T e669b630ee114961ad830647ab90abc3 P e1052c249dde44e6901a8b7840c6e1bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 19:03:07.178797 31156 raft_consensus.cc:482] T e669b630ee114961ad830647ab90abc3 P e1052c249dde44e6901a8b7840c6e1bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 19:03:07.178850 31156 raft_consensus.cc:3037] T e669b630ee114961ad830647ab90abc3 P e1052c249dde44e6901a8b7840c6e1bb [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:03:07.180177 31156 raft_consensus.cc:504] T e669b630ee114961ad830647ab90abc3 P e1052c249dde44e6901a8b7840c6e1bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1052c249dde44e6901a8b7840c6e1bb" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 45197 } } peers { permanent_uuid: "0b24c35b827f4b0badf0b66aec15cae9" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 36839 } } peers { permanent_uuid: "a2336f52031842798c0dabb12b7e06e6" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 40165 } }
I20260811 19:03:07.180591 31156 leader_election.cc:288] T e669b630ee114961ad830647ab90abc3 P e1052c249dde44e6901a8b7840c6e1bb [CANDIDATE]: Term 1 election: Requested vote from peers 0b24c35b827f4b0badf0b66aec15cae9 (127.13.109.1:36839), a2336f52031842798c0dabb12b7e06e6 (127.13.109.3:40165)
I20260811 19:03:07.181128 31080 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e669b630ee114961ad830647ab90abc3" candidate_uuid: "e1052c249dde44e6901a8b7840c6e1bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2336f52031842798c0dabb12b7e06e6"
I20260811 19:03:07.181157 30777 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e669b630ee114961ad830647ab90abc3" candidate_uuid: "e1052c249dde44e6901a8b7840c6e1bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b24c35b827f4b0badf0b66aec15cae9"
I20260811 19:03:07.181288 30777 raft_consensus.cc:3037] T e669b630ee114961ad830647ab90abc3 P 0b24c35b827f4b0badf0b66aec15cae9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:03:07.181288 31080 raft_consensus.cc:3037] T e669b630ee114961ad830647ab90abc3 P a2336f52031842798c0dabb12b7e06e6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:03:07.182765 31080 raft_consensus.cc:2454] T e669b630ee114961ad830647ab90abc3 P a2336f52031842798c0dabb12b7e06e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1052c249dde44e6901a8b7840c6e1bb in term 1.
I20260811 19:03:07.182765 30777 raft_consensus.cc:2454] T e669b630ee114961ad830647ab90abc3 P 0b24c35b827f4b0badf0b66aec15cae9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1052c249dde44e6901a8b7840c6e1bb in term 1.
I20260811 19:03:07.183167 30851 leader_election.cc:302] T e669b630ee114961ad830647ab90abc3 P e1052c249dde44e6901a8b7840c6e1bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2336f52031842798c0dabb12b7e06e6, e1052c249dde44e6901a8b7840c6e1bb; no voters:
I20260811 19:03:07.183357 31156 raft_consensus.cc:2781] T e669b630ee114961ad830647ab90abc3 P e1052c249dde44e6901a8b7840c6e1bb [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:03:07.183588 31156 raft_consensus.cc:686] T e669b630ee114961ad830647ab90abc3 P e1052c249dde44e6901a8b7840c6e1bb [term 1 LEADER]: Becoming Leader. State: Replica: e1052c249dde44e6901a8b7840c6e1bb, State: Running, Role: LEADER
I20260811 19:03:07.183988 31156 consensus_queue.cc:237] T e669b630ee114961ad830647ab90abc3 P e1052c249dde44e6901a8b7840c6e1bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1052c249dde44e6901a8b7840c6e1bb" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 45197 } } peers { permanent_uuid: "0b24c35b827f4b0badf0b66aec15cae9" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 36839 } } peers { permanent_uuid: "a2336f52031842798c0dabb12b7e06e6" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 40165 } }
I20260811 19:03:07.187531 30617 catalog_manager.cc:5495] T e669b630ee114961ad830647ab90abc3 P e1052c249dde44e6901a8b7840c6e1bb reported cstate change: term changed from 0 to 1, leader changed from <none> to e1052c249dde44e6901a8b7840c6e1bb (127.13.109.2). New cstate: current_term: 1 leader_uuid: "e1052c249dde44e6901a8b7840c6e1bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1052c249dde44e6901a8b7840c6e1bb" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 45197 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0b24c35b827f4b0badf0b66aec15cae9" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 36839 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2336f52031842798c0dabb12b7e06e6" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 40165 } health_report { overall_health: UNKNOWN } } }
I20260811 19:03:07.308266 31169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:07.303689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55104 (local address 127.13.109.2:45197)
0811 19:03:07.304000 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:07.304003 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:07.304020 (+ 17us) server_negotiation.cc:407] Connection header received
0811 19:03:07.304173 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:07.304176 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:07.304237 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:07.304393 (+ 156us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:03:07.304740 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:07.305549 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:07.306568 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:07.306816 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:07.306912 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:07.307283 (+ 371us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:03:07.307788 (+ 505us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:03:07.307899 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:07.308055 (+ 156us) server_negotiation.cc:299] Negotiation successful
0811 19:03:07.308118 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":84,"threads_started":1}
I20260811 19:03:07.315120 31080 raft_consensus.cc:1261] T e669b630ee114961ad830647ab90abc3 P a2336f52031842798c0dabb12b7e06e6 [term 1 FOLLOWER]: Refusing update from remote peer e1052c249dde44e6901a8b7840c6e1bb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:03:07.315120 30777 raft_consensus.cc:1261] T e669b630ee114961ad830647ab90abc3 P 0b24c35b827f4b0badf0b66aec15cae9 [term 1 FOLLOWER]: Refusing update from remote peer e1052c249dde44e6901a8b7840c6e1bb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:03:07.316119 31156 consensus_queue.cc:1035] T e669b630ee114961ad830647ab90abc3 P e1052c249dde44e6901a8b7840c6e1bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b24c35b827f4b0badf0b66aec15cae9" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 36839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:03:07.316433 31165 consensus_queue.cc:1035] T e669b630ee114961ad830647ab90abc3 P e1052c249dde44e6901a8b7840c6e1bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2336f52031842798c0dabb12b7e06e6" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 40165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:03:07.324855 31170 mvcc.cc:205] Tried to move back new op lower bound from 7317401548033490944 to 7317401547508580352. Current Snapshot: MvccSnapshot[applied={T|T < 7317401548033490944}]
I20260811 19:03:07.326095 31173 mvcc.cc:205] Tried to move back new op lower bound from 7317401548033490944 to 7317401547508580352. Current Snapshot: MvccSnapshot[applied={T|T < 7317401548033490944}]
I20260811 19:03:07.334654 30616 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50436:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:03:07.334848 30616 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50436:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:03:07.338007 30616 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7d3cc91d778246cdb59c93831c15d546: Sending DeleteTablet for 3 replicas of tablet e669b630ee114961ad830647ab90abc3
I20260811 19:03:07.339120 30757 tablet_service.cc:1514] Processing DeleteTablet for tablet e669b630ee114961ad830647ab90abc3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:03:07 UTC) from {username='kudu', principal='kudu/127.13.109.62@KRBTEST.COM'} at 127.0.0.1:58208
I20260811 19:03:07.339120 30905 tablet_service.cc:1514] Processing DeleteTablet for tablet e669b630ee114961ad830647ab90abc3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:03:07 UTC) from {username='kudu', principal='kudu/127.13.109.62@KRBTEST.COM'} at 127.0.0.1:55102
I20260811 19:03:07.339442 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 30685
I20260811 19:03:07.339727 31058 tablet_service.cc:1514] Processing DeleteTablet for tablet e669b630ee114961ad830647ab90abc3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:03:07 UTC) from {username='kudu', principal='kudu/127.13.109.62@KRBTEST.COM'} at 127.0.0.1:47822
I20260811 19:03:07.341964 31188 tablet_replica.cc:330] T e669b630ee114961ad830647ab90abc3 P a2336f52031842798c0dabb12b7e06e6: stopping tablet replica
I20260811 19:03:07.342466 31188 raft_consensus.cc:2229] T e669b630ee114961ad830647ab90abc3 P a2336f52031842798c0dabb12b7e06e6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:03:07.342965 31188 raft_consensus.cc:2258] T e669b630ee114961ad830647ab90abc3 P a2336f52031842798c0dabb12b7e06e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:03:07.344828 31188 ts_tablet_manager.cc:1905] T e669b630ee114961ad830647ab90abc3 P a2336f52031842798c0dabb12b7e06e6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:03:07.349089 31188 ts_tablet_manager.cc:1918] T e669b630ee114961ad830647ab90abc3 P a2336f52031842798c0dabb12b7e06e6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:03:07.349261 31188 log.cc:1196] T e669b630ee114961ad830647ab90abc3 P a2336f52031842798c0dabb12b7e06e6: Deleting WAL directory at /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-2/wal/wals/e669b630ee114961ad830647ab90abc3
I20260811 19:03:07.349623 31188 ts_tablet_manager.cc:1939] T e669b630ee114961ad830647ab90abc3 P a2336f52031842798c0dabb12b7e06e6: Deleting consensus metadata
I20260811 19:03:07.349818 31186 tablet_replica.cc:330] T e669b630ee114961ad830647ab90abc3 P e1052c249dde44e6901a8b7840c6e1bb: stopping tablet replica
I20260811 19:03:07.350428 31186 raft_consensus.cc:2229] T e669b630ee114961ad830647ab90abc3 P e1052c249dde44e6901a8b7840c6e1bb [term 1 LEADER]: Raft consensus shutting down.
I20260811 19:03:07.351046 31186 raft_consensus.cc:2258] T e669b630ee114961ad830647ab90abc3 P e1052c249dde44e6901a8b7840c6e1bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:03:07.352370 30600 catalog_manager.cc:4841] TS a2336f52031842798c0dabb12b7e06e6 (127.13.109.3:40165): tablet e669b630ee114961ad830647ab90abc3 (table test-table [id=ebc7efa4b6cb46b2ae51391c64a0d047]) successfully deleted
W20260811 19:03:07.352701 30600 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.109.1:36839 (error 108)
I20260811 19:03:07.353480 31186 ts_tablet_manager.cc:1905] T e669b630ee114961ad830647ab90abc3 P e1052c249dde44e6901a8b7840c6e1bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:03:07.354040 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 30833
I20260811 19:03:07.354074 31190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:07.353451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.1:36839 (local address 127.0.0.1:58214)
0811 19:03:07.353849 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:07.353941 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.1:36839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":79,"threads_started":1}
W20260811 19:03:07.358124 30600 catalog_manager.cc:4584] TS 0b24c35b827f4b0badf0b66aec15cae9 (127.13.109.1:36839): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e669b630ee114961ad830647ab90abc3: Network error: Client connection negotiation failed: client connection to 127.13.109.1:36839: connect: Connection refused (error 111)
I20260811 19:03:07.364687 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 30986
I20260811 19:03:07.367221 31190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:07.366407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.2:45197 (local address 127.0.0.1:55110)
0811 19:03:07.366994 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:07.367086 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.2:45197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":451}
I20260811 19:03:07.374205 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 30583
I20260811 19:03:27.384740 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.62:36875
--webserver_interface=127.13.109.62
--webserver_port=42837
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab
--principal=kudu/127.13.109.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:32993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.109.62:36875
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:03:27.512048 31191 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:03:27.512353 31191 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:03:27.512398 31191 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:03:27.516597 31191 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 19:03:27.516673 31191 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 19:03:27.516690 31191 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 19:03:27.516705 31191 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:03:27.516723 31191 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:03:27.516742 31191 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:03:27.516799 31191 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:03:27.516839 31191 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:03:27.522598 31191 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:32993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.13.109.62:36875
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab
--principal=kudu/127.13.109.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.62:36875
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.109.62
--webserver_port=42837
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31191
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:03:27.523900 31191 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:03:27.525049 31191 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:03:27.540444 31198 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:03:27.540444 31199 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:03:27.540445 31201 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:03:27.540498 31191 server_base.cc:1034] running on GCE node
I20260811 19:03:27.541481 31191 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:03:27.542887 31191 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:03:27.544090 31191 hybrid_clock.cc:648] HybridClock initialized: now 1786475007544075 us; error 37 us; skew 500 ppm
Aug 11 19:03:27 dist-test-slave-0kls krb5kdc[30569](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475007, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:03:27.547060 31191 init.cc:377] Logged in from keytab as kudu/127.13.109.62@KRBTEST.COM (short username kudu)
I20260811 19:03:27.548115 31191 webserver.cc:460] Webserver started at http://127.13.109.62:42837/ using document root <none> and password file <none>
I20260811 19:03:27.548756 31191 fs_manager.cc:359] Metadata directory not provided
I20260811 19:03:27.548879 31191 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:03:27.553272 31191 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 19:03:27.555738 31208 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:03:27.557056 31191 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 19:03:27.557242 31191 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/master-0/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/master-0/wal
uuid: "7d3cc91d778246cdb59c93831c15d546"
format_stamp: "Formatted at 2026-08-11 19:03:05 on dist-test-slave-0kls"
server_key: "e47d5437894a1d90f80f2212a32b4840"
server_key_iv: "96c4a8f843d3b7c7b8f8538711c6ed69"
server_key_version: "encryptionkey@0"
I20260811 19:03:27.557976 31191 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:03:27.569298 31191 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:03:27.572388 31191 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:03:27.572613 31191 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:03:27.580593 31191 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.62:36875
I20260811 19:03:27.580914 31266 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.62:36875 every 8 connection(s)
I20260811 19:03:27.581827 31191 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
I20260811 19:03:27.584532 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 31191
I20260811 19:03:27.584967 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.1:36839
--local_ip_for_outbound_sockets=127.13.109.1
--tserver_master_addrs=127.13.109.62:36875
--webserver_port=33679
--webserver_interface=127.13.109.1
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.1.keytab
--principal=kudu/127.13.109.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:32993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 19:03:27.586580 31268 sys_catalog.cc:263] Verifying existing consensus state
I20260811 19:03:27.588320 31268 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d3cc91d778246cdb59c93831c15d546: Bootstrap starting.
I20260811 19:03:27.606616 31268 log.cc:824] T 00000000000000000000000000000000 P 7d3cc91d778246cdb59c93831c15d546: Log is configured to *not* fsync() on all Append() calls
I20260811 19:03:27.615885 31268 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d3cc91d778246cdb59c93831c15d546: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 19:03:27.616362 31268 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d3cc91d778246cdb59c93831c15d546: Bootstrap complete.
I20260811 19:03:27.620946 31268 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7d3cc91d778246cdb59c93831c15d546 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d3cc91d778246cdb59c93831c15d546" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 36875 } }
I20260811 19:03:27.621412 31268 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7d3cc91d778246cdb59c93831c15d546 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d3cc91d778246cdb59c93831c15d546, State: Initialized, Role: FOLLOWER
I20260811 19:03:27.621997 31268 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7d3cc91d778246cdb59c93831c15d546 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d3cc91d778246cdb59c93831c15d546" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 36875 } }
I20260811 19:03:27.622152 31268 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7d3cc91d778246cdb59c93831c15d546 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:03:27.622251 31268 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7d3cc91d778246cdb59c93831c15d546 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:03:27.622377 31268 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7d3cc91d778246cdb59c93831c15d546 [term 1 FOLLOWER]: Advancing to term 2
I20260811 19:03:27.624504 31268 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7d3cc91d778246cdb59c93831c15d546 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d3cc91d778246cdb59c93831c15d546" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 36875 } }
I20260811 19:03:27.624881 31268 leader_election.cc:302] T 00000000000000000000000000000000 P 7d3cc91d778246cdb59c93831c15d546 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7d3cc91d778246cdb59c93831c15d546; no voters:
I20260811 19:03:27.625258 31268 leader_election.cc:288] T 00000000000000000000000000000000 P 7d3cc91d778246cdb59c93831c15d546 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 19:03:27.625355 31273 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7d3cc91d778246cdb59c93831c15d546 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 19:03:27.625554 31273 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7d3cc91d778246cdb59c93831c15d546 [term 2 LEADER]: Becoming Leader. State: Replica: 7d3cc91d778246cdb59c93831c15d546, State: Running, Role: LEADER
I20260811 19:03:27.625958 31273 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7d3cc91d778246cdb59c93831c15d546 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d3cc91d778246cdb59c93831c15d546" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 36875 } }
I20260811 19:03:27.626684 31268 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7d3cc91d778246cdb59c93831c15d546 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:03:27.627638 31274 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d3cc91d778246cdb59c93831c15d546 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "7d3cc91d778246cdb59c93831c15d546" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d3cc91d778246cdb59c93831c15d546" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 36875 } } }
I20260811 19:03:27.627686 31275 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d3cc91d778246cdb59c93831c15d546 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7d3cc91d778246cdb59c93831c15d546. Latest consensus state: current_term: 2 leader_uuid: "7d3cc91d778246cdb59c93831c15d546" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d3cc91d778246cdb59c93831c15d546" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 36875 } } }
I20260811 19:03:27.627780 31274 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d3cc91d778246cdb59c93831c15d546 [sys.catalog]: This master's current role is: LEADER
I20260811 19:03:27.627807 31275 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d3cc91d778246cdb59c93831c15d546 [sys.catalog]: This master's current role is: LEADER
I20260811 19:03:27.632124 31290 catalog_manager.cc:1259] Loaded cluster ID: aa1f3e5656714d4fb6a72e293346f4f4
I20260811 19:03:27.632213 31290 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7d3cc91d778246cdb59c93831c15d546: loading cluster ID for follower catalog manager: success
I20260811 19:03:27.634128 31290 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7d3cc91d778246cdb59c93831c15d546: acquiring CA information for follower catalog manager: success
I20260811 19:03:27.635381 31290 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7d3cc91d778246cdb59c93831c15d546: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 19:03:27.635535 31282 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:03:27.638074 31282 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:03:27.638556 31282 catalog_manager.cc:1259] Loaded cluster ID: aa1f3e5656714d4fb6a72e293346f4f4
I20260811 19:03:27.638621 31282 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:03:27.639299 31282 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:03:27.639801 31282 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 7d3cc91d778246cdb59c93831c15d546: Loaded TSK: 0
I20260811 19:03:27.655563 31282 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7d3cc91d778246cdb59c93831c15d546: Generated new TSK 1
I20260811 19:03:27.655897 31282 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 19:03:27.715145 31271 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:03:27.715438 31271 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:03:27.715483 31271 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:03:27.719789 31271 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:03:27.719868 31271 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:03:27.719931 31271 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.1
I20260811 19:03:27.725606 31271 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:32993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.1.keytab
--principal=kudu/127.13.109.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.1:36839
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.109.1
--webserver_port=33679
--tserver_master_addrs=127.13.109.62:36875
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31271
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.1
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:03:27.726823 31271 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:03:27.728008 31271 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:03:27.739851 31300 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:03:27.739848 31301 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:03:27.740208 31271 server_base.cc:1034] running on GCE node
W20260811 19:03:27.739851 31303 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:03:27.740706 31271 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:03:27.741456 31271 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:03:27.742636 31271 hybrid_clock.cc:648] HybridClock initialized: now 1786475007742606 us; error 47 us; skew 500 ppm
Aug 11 19:03:27 dist-test-slave-0kls krb5kdc[30569](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475007, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:03:27.745441 31271 init.cc:377] Logged in from keytab as kudu/127.13.109.1@KRBTEST.COM (short username kudu)
I20260811 19:03:27.746488 31271 webserver.cc:460] Webserver started at http://127.13.109.1:33679/ using document root <none> and password file <none>
I20260811 19:03:27.747084 31271 fs_manager.cc:359] Metadata directory not provided
I20260811 19:03:27.747164 31271 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:03:27.751629 31271 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 19:03:27.754267 31312 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:03:27.755746 31271 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 19:03:27.755863 31271 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-0/wal
uuid: "0b24c35b827f4b0badf0b66aec15cae9"
format_stamp: "Formatted at 2026-08-11 19:03:06 on dist-test-slave-0kls"
server_key: "1cd0cfa39f884ff13a23f2075b5f2bfa"
server_key_iv: "d92e0b1842bb9ef1b0a5661f58bebd7a"
server_key_version: "encryptionkey@0"
I20260811 19:03:27.756503 31271 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:03:27.785369 31271 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:03:27.789168 31271 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:03:27.789417 31271 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:03:27.790215 31271 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:03:27.791522 31321 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 19:03:27.796351 31271 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 19:03:27.796451 31271 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260811 19:03:27.796561 31271 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 19:03:27.798780 31271 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 19:03:27.798846 31271 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.002s sys 0.000s
I20260811 19:03:27.799260 31321 tablet_bootstrap.cc:492] T e669b630ee114961ad830647ab90abc3 P 0b24c35b827f4b0badf0b66aec15cae9: Bootstrap starting.
I20260811 19:03:27.811131 31271 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.1:36839
I20260811 19:03:27.811228 31436 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.1:36839 every 8 connection(s)
I20260811 19:03:27.812252 31271 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
I20260811 19:03:27.813711 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 31271
I20260811 19:03:27.814291 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.2:45197
--local_ip_for_outbound_sockets=127.13.109.2
--tserver_master_addrs=127.13.109.62:36875
--webserver_port=36677
--webserver_interface=127.13.109.2
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.2.keytab
--principal=kudu/127.13.109.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:32993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 19:03:27.820531 31321 log.cc:824] T e669b630ee114961ad830647ab90abc3 P 0b24c35b827f4b0badf0b66aec15cae9: Log is configured to *not* fsync() on all Append() calls
Aug 11 19:03:27 dist-test-slave-0kls krb5kdc[30569](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475007, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.1@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
I20260811 19:03:27.831446 31440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:27.814496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:36875 (local address 127.13.109.1:36123)
0811 19:03:27.815467 (+ 971us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:27.815520 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 19:03:27.816451 (+ 931us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:03:27.817999 (+ 1548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:03:27.818005 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:03:27.818358 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:03:27.818836 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:03:27.818850 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:03:27.820778 (+ 1928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:03:27.820782 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:03:27.822013 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:03:27.822025 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:03:27.822860 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:03:27.822864 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:03:27.823058 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:27.823767 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:03:27.823785 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:03:27.825513 (+ 1728us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:03:27.827908 (+ 2395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:03:27.827913 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:03:27.827924 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:03:27.828253 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:03:27.828666 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:03:27.828669 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:03:27.828671 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:03:27.828815 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:03:27.829546 (+ 731us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:03:27.829553 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:03:27.829886 (+ 333us) client_negotiation.cc:769] Sending connection context
0811 19:03:27.830084 (+ 198us) client_negotiation.cc:240] Negotiation successful
0811 19:03:27.830461 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":758,"thread_start_us":139,"threads_started":1}
I20260811 19:03:27.831427 31439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:27.814464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.1:36123 (local address 127.13.109.62:36875)
0811 19:03:27.814996 (+ 532us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:27.815017 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:27.815655 (+ 638us) server_negotiation.cc:407] Connection header received
0811 19:03:27.817001 (+ 1346us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:27.817019 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:27.817444 (+ 425us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:27.817933 (+ 489us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:27.819032 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:27.820581 (+ 1549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:27.822206 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:27.822684 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:27.822875 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:27.825737 (+ 2862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:03:27.825743 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:03:27.825755 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:03:27.825834 (+ 79us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:03:27.827706 (+ 1872us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:27.828381 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:27.828388 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:27.828394 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:27.828550 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:27.828951 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:27.828955 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:27.828957 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:27.829421 (+ 464us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:03:27.829572 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:27.830215 (+ 643us) server_negotiation.cc:299] Negotiation successful
0811 19:03:27.830492 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":135,"threads_started":1}
I20260811 19:03:27.831791 31321 tablet_bootstrap.cc:492] T e669b630ee114961ad830647ab90abc3 P 0b24c35b827f4b0badf0b66aec15cae9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 19:03:27.832860 31321 tablet_bootstrap.cc:492] T e669b630ee114961ad830647ab90abc3 P 0b24c35b827f4b0badf0b66aec15cae9: Bootstrap complete.
I20260811 19:03:27.834522 31321 ts_tablet_manager.cc:1397] T e669b630ee114961ad830647ab90abc3 P 0b24c35b827f4b0badf0b66aec15cae9: Time spent bootstrapping tablet: real 0.036s user 0.022s sys 0.012s
I20260811 19:03:27.834787 31437 heartbeater.cc:347] Connected to a master server at 127.13.109.62:36875
I20260811 19:03:27.835050 31437 heartbeater.cc:464] Registering TS with master...
I20260811 19:03:27.835624 31437 heartbeater.cc:511] Master 127.13.109.62:36875 requested a full tablet report, sending...
I20260811 19:03:27.837890 31321 raft_consensus.cc:348] T e669b630ee114961ad830647ab90abc3 P 0b24c35b827f4b0badf0b66aec15cae9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1052c249dde44e6901a8b7840c6e1bb" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 45197 } } peers { permanent_uuid: "0b24c35b827f4b0badf0b66aec15cae9" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 36839 } } peers { permanent_uuid: "a2336f52031842798c0dabb12b7e06e6" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 40165 } }
I20260811 19:03:27.838522 31228 ts_manager.cc:194] Registered new tserver with Master: 0b24c35b827f4b0badf0b66aec15cae9 (127.13.109.1:36839)
I20260811 19:03:27.838549 31321 raft_consensus.cc:729] T e669b630ee114961ad830647ab90abc3 P 0b24c35b827f4b0badf0b66aec15cae9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b24c35b827f4b0badf0b66aec15cae9, State: Initialized, Role: FOLLOWER
I20260811 19:03:27.839098 31321 consensus_queue.cc:260] T e669b630ee114961ad830647ab90abc3 P 0b24c35b827f4b0badf0b66aec15cae9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1052c249dde44e6901a8b7840c6e1bb" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 45197 } } peers { permanent_uuid: "0b24c35b827f4b0badf0b66aec15cae9" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 36839 } } peers { permanent_uuid: "a2336f52031842798c0dabb12b7e06e6" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 40165 } }
I20260811 19:03:27.840495 31321 ts_tablet_manager.cc:1428] T e669b630ee114961ad830647ab90abc3 P 0b24c35b827f4b0badf0b66aec15cae9: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260811 19:03:27.842935 31228 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.109.1@KRBTEST.COM'} at 127.13.109.1:36123
I20260811 19:03:27.845817 31437 heartbeater.cc:503] Master 127.13.109.62:36875 was elected leader, sending a full tablet report...
Aug 11 19:03:27 dist-test-slave-0kls krb5kdc[30569](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475007, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.62@KRBTEST.COM for kudu/127.13.109.1@KRBTEST.COM
I20260811 19:03:27.854393 31445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:27.842243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.1:36839 (local address 127.0.0.1:40842)
0811 19:03:27.842981 (+ 738us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:27.843039 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 19:03:27.843226 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:03:27.844367 (+ 1141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:03:27.844374 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:03:27.844855 (+ 481us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:03:27.845011 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:03:27.845019 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:03:27.846357 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:03:27.846360 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:03:27.847258 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:03:27.847270 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:03:27.848041 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:03:27.848044 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:03:27.848127 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:27.848729 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:03:27.848742 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:03:27.850207 (+ 1465us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:03:27.852379 (+ 2172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:03:27.852388 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:03:27.852393 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:03:27.852659 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:03:27.852994 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:03:27.852996 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:03:27.852997 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:03:27.853080 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:03:27.853700 (+ 620us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:03:27.853709 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:03:27.853913 (+ 204us) client_negotiation.cc:769] Sending connection context
0811 19:03:27.854040 (+ 127us) client_negotiation.cc:240] Negotiation successful
0811 19:03:27.854232 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":114,"threads_started":1}
I20260811 19:03:27.854557 31446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:27.842368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40842 (local address 127.13.109.1:36839)
0811 19:03:27.843877 (+ 1509us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:27.843888 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:27.843910 (+ 22us) server_negotiation.cc:407] Connection header received
0811 19:03:27.843969 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:27.843973 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:27.844194 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:27.844337 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:27.845168 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:27.846219 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:27.847404 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:27.847907 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:27.848015 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:27.850363 (+ 2348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:03:27.850374 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:03:27.850389 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:03:27.850417 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:03:27.852246 (+ 1829us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:27.852782 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:27.852789 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:27.852790 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:27.852894 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:27.853174 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:27.853177 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:27.853179 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:27.853532 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:03:27.853700 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:27.854161 (+ 461us) server_negotiation.cc:299] Negotiation successful
0811 19:03:27.854379 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1401,"thread_start_us":126,"threads_started":1}
I20260811 19:03:27.855656 31366 tablet_service.cc:1514] Processing DeleteTablet for tablet e669b630ee114961ad830647ab90abc3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:03:07 UTC) from {username='kudu', principal='kudu/127.13.109.62@KRBTEST.COM'} at 127.0.0.1:40842
I20260811 19:03:27.856614 31447 tablet_replica.cc:330] stopping tablet replica
I20260811 19:03:27.856956 31447 raft_consensus.cc:2229] T e669b630ee114961ad830647ab90abc3 P 0b24c35b827f4b0badf0b66aec15cae9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:03:27.857236 31447 raft_consensus.cc:2258] T e669b630ee114961ad830647ab90abc3 P 0b24c35b827f4b0badf0b66aec15cae9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:03:27.858398 31447 ts_tablet_manager.cc:1905] T e669b630ee114961ad830647ab90abc3 P 0b24c35b827f4b0badf0b66aec15cae9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:03:27.862965 31447 ts_tablet_manager.cc:1918] T e669b630ee114961ad830647ab90abc3 P 0b24c35b827f4b0badf0b66aec15cae9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:03:27.863077 31447 log.cc:1196] T e669b630ee114961ad830647ab90abc3 P 0b24c35b827f4b0badf0b66aec15cae9: Deleting WAL directory at /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-0/wal/wals/e669b630ee114961ad830647ab90abc3
I20260811 19:03:27.863376 31447 ts_tablet_manager.cc:1939] T e669b630ee114961ad830647ab90abc3 P 0b24c35b827f4b0badf0b66aec15cae9: Deleting consensus metadata
I20260811 19:03:27.865437 31210 catalog_manager.cc:4841] TS 0b24c35b827f4b0badf0b66aec15cae9 (127.13.109.1:36839): tablet e669b630ee114961ad830647ab90abc3 (table test-table [id=ebc7efa4b6cb46b2ae51391c64a0d047]) successfully deleted
W20260811 19:03:27.951804 31441 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:03:27.952129 31441 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:03:27.952178 31441 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:03:27.956393 31441 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:03:27.956475 31441 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:03:27.956538 31441 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.2
I20260811 19:03:27.962750 31441 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:32993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.2.keytab
--principal=kudu/127.13.109.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.2:45197
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.109.2
--webserver_port=36677
--tserver_master_addrs=127.13.109.62:36875
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31441
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.2
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:03:27.964074 31441 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:03:27.965204 31441 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:03:27.975773 31453 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:03:27.975899 31457 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:03:27.975773 31455 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:03:27.976213 31441 server_base.cc:1034] running on GCE node
I20260811 19:03:27.976735 31441 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:03:27.977412 31441 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:03:27.978618 31441 hybrid_clock.cc:648] HybridClock initialized: now 1786475007978588 us; error 43 us; skew 500 ppm
Aug 11 19:03:27 dist-test-slave-0kls krb5kdc[30569](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475007, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:03:27.981588 31441 init.cc:377] Logged in from keytab as kudu/127.13.109.2@KRBTEST.COM (short username kudu)
I20260811 19:03:27.982630 31441 webserver.cc:460] Webserver started at http://127.13.109.2:36677/ using document root <none> and password file <none>
I20260811 19:03:27.983265 31441 fs_manager.cc:359] Metadata directory not provided
I20260811 19:03:27.983392 31441 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:03:27.987763 31441 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.002s
I20260811 19:03:27.990345 31465 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:03:27.991750 31441 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 19:03:27.991921 31441 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-1/wal
uuid: "e1052c249dde44e6901a8b7840c6e1bb"
format_stamp: "Formatted at 2026-08-11 19:03:06 on dist-test-slave-0kls"
server_key: "6383c25d81e363f2d5dab237d58d0b1d"
server_key_iv: "9d9dc05dcc7917ea22074ebcd087da87"
server_key_version: "encryptionkey@0"
I20260811 19:03:27.992553 31441 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:03:28.015908 31441 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:03:28.019479 31441 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:03:28.019712 31441 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:03:28.020383 31441 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:03:28.021544 31474 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 19:03:28.026048 31441 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 19:03:28.026151 31441 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260811 19:03:28.026248 31441 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 19:03:28.028463 31441 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 19:03:28.028563 31441 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260811 19:03:28.028843 31474 tablet_bootstrap.cc:492] T e669b630ee114961ad830647ab90abc3 P e1052c249dde44e6901a8b7840c6e1bb: Bootstrap starting.
I20260811 19:03:28.036147 31474 log.cc:824] T e669b630ee114961ad830647ab90abc3 P e1052c249dde44e6901a8b7840c6e1bb: Log is configured to *not* fsync() on all Append() calls
I20260811 19:03:28.039852 31441 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.2:45197
I20260811 19:03:28.039866 31594 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.2:45197 every 8 connection(s)
I20260811 19:03:28.040819 31441 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
I20260811 19:03:28.045365 31474 tablet_bootstrap.cc:492] T e669b630ee114961ad830647ab90abc3 P e1052c249dde44e6901a8b7840c6e1bb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 19:03:28.045842 31474 tablet_bootstrap.cc:492] T e669b630ee114961ad830647ab90abc3 P e1052c249dde44e6901a8b7840c6e1bb: Bootstrap complete.
I20260811 19:03:28.046231 31474 ts_tablet_manager.cc:1397] T e669b630ee114961ad830647ab90abc3 P e1052c249dde44e6901a8b7840c6e1bb: Time spent bootstrapping tablet: real 0.018s user 0.014s sys 0.003s
I20260811 19:03:28.047636 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 31441
I20260811 19:03:28.048113 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.3:40165
--local_ip_for_outbound_sockets=127.13.109.3
--tserver_master_addrs=127.13.109.62:36875
--webserver_port=37157
--webserver_interface=127.13.109.3
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.3.keytab
--principal=kudu/127.13.109.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:32993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 19:03:28.050483 31474 raft_consensus.cc:348] T e669b630ee114961ad830647ab90abc3 P e1052c249dde44e6901a8b7840c6e1bb [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1052c249dde44e6901a8b7840c6e1bb" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 45197 } } peers { permanent_uuid: "0b24c35b827f4b0badf0b66aec15cae9" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 36839 } } peers { permanent_uuid: "a2336f52031842798c0dabb12b7e06e6" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 40165 } }
Aug 11 19:03:28 dist-test-slave-0kls krb5kdc[30569](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475007, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.2@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
I20260811 19:03:28.051070 31474 raft_consensus.cc:729] T e669b630ee114961ad830647ab90abc3 P e1052c249dde44e6901a8b7840c6e1bb [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1052c249dde44e6901a8b7840c6e1bb, State: Initialized, Role: FOLLOWER
I20260811 19:03:28.051586 31474 consensus_queue.cc:260] T e669b630ee114961ad830647ab90abc3 P e1052c249dde44e6901a8b7840c6e1bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1052c249dde44e6901a8b7840c6e1bb" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 45197 } } peers { permanent_uuid: "0b24c35b827f4b0badf0b66aec15cae9" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 36839 } } peers { permanent_uuid: "a2336f52031842798c0dabb12b7e06e6" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 40165 } }
I20260811 19:03:28.052784 31474 ts_tablet_manager.cc:1428] T e669b630ee114961ad830647ab90abc3 P e1052c249dde44e6901a8b7840c6e1bb: Time spent starting tablet: real 0.006s user 0.003s sys 0.004s
I20260811 19:03:28.056234 31439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:28.042898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.2:35543 (local address 127.13.109.62:36875)
0811 19:03:28.043076 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:28.043081 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:28.043769 (+ 688us) server_negotiation.cc:407] Connection header received
0811 19:03:28.044755 (+ 986us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:28.044759 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:28.044821 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:28.044933 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:28.046252 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:28.047025 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:28.048318 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:28.048615 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:28.048681 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:28.051931 (+ 3250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:03:28.051936 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:03:28.051939 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:03:28.051970 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:03:28.053607 (+ 1637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:28.054354 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:28.054357 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:28.054358 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:28.054476 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:28.054925 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:28.054928 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:28.054929 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:28.055189 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:03:28.055295 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:28.055968 (+ 673us) server_negotiation.cc:299] Negotiation successful
0811 19:03:28.056087 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 19:03:28.057523 31599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:28.043050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:36875 (local address 127.13.109.2:35543)
0811 19:03:28.043592 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:28.043646 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 19:03:28.044542 (+ 896us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:03:28.045072 (+ 530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:03:28.045079 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:03:28.045445 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:03:28.046078 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:03:28.046094 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:03:28.047204 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:03:28.047207 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:03:28.048140 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:03:28.048150 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:03:28.048839 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:03:28.048842 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:03:28.048984 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:28.049633 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:03:28.049650 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:03:28.051708 (+ 2058us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:03:28.053737 (+ 2029us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:03:28.053746 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:03:28.053764 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:03:28.054236 (+ 472us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:03:28.054596 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:03:28.054602 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:03:28.054607 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:03:28.054816 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:03:28.055314 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:03:28.055323 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:03:28.055684 (+ 361us) client_negotiation.cc:769] Sending connection context
0811 19:03:28.055922 (+ 238us) client_negotiation.cc:240] Negotiation successful
0811 19:03:28.056331 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":173,"threads_started":1}
I20260811 19:03:28.058873 31595 heartbeater.cc:347] Connected to a master server at 127.13.109.62:36875
I20260811 19:03:28.059129 31595 heartbeater.cc:464] Registering TS with master...
I20260811 19:03:28.059696 31595 heartbeater.cc:511] Master 127.13.109.62:36875 requested a full tablet report, sending...
I20260811 19:03:28.061689 31228 ts_manager.cc:194] Registered new tserver with Master: e1052c249dde44e6901a8b7840c6e1bb (127.13.109.2:45197)
I20260811 19:03:28.063122 31228 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.109.2@KRBTEST.COM'} at 127.13.109.2:35543
I20260811 19:03:28.065546 31595 heartbeater.cc:503] Master 127.13.109.62:36875 was elected leader, sending a full tablet report...
Aug 11 19:03:28 dist-test-slave-0kls krb5kdc[30569](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475007, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.62@KRBTEST.COM for kudu/127.13.109.2@KRBTEST.COM
I20260811 19:03:28.073688 31445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:28.062835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.2:45197 (local address 127.0.0.1:37076)
0811 19:03:28.063019 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:28.063033 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 19:03:28.063107 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:03:28.064088 (+ 981us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:03:28.064092 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:03:28.064305 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:03:28.064414 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:03:28.064421 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:03:28.065822 (+ 1401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:03:28.065827 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:03:28.066705 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:03:28.066716 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:03:28.067477 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:03:28.067481 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:03:28.067585 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:28.068075 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:03:28.068094 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:03:28.069608 (+ 1514us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:03:28.071814 (+ 2206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:03:28.071818 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:03:28.071825 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:03:28.072087 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:03:28.072447 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:03:28.072450 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:03:28.072452 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:03:28.072553 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:03:28.073045 (+ 492us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:03:28.073048 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:03:28.073205 (+ 157us) client_negotiation.cc:769] Sending connection context
0811 19:03:28.073319 (+ 114us) client_negotiation.cc:240] Negotiation successful
0811 19:03:28.073481 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111}
I20260811 19:03:28.073714 31602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:28.062953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37076 (local address 127.13.109.2:45197)
0811 19:03:28.063586 (+ 633us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:28.063599 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:28.063623 (+ 24us) server_negotiation.cc:407] Connection header received
0811 19:03:28.063700 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:28.063706 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:28.063934 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:28.064074 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:28.064544 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:28.065647 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:28.066961 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:28.067361 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:28.067448 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:28.069780 (+ 2332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:03:28.069855 (+ 75us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:03:28.069874 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:03:28.069905 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:03:28.071695 (+ 1790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:28.072243 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:28.072249 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:28.072250 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:28.072341 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:28.072662 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:28.072665 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:28.072666 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:28.072937 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:03:28.073081 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:28.073415 (+ 334us) server_negotiation.cc:299] Negotiation successful
0811 19:03:28.073586 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":535,"thread_start_us":92,"threads_started":1}
I20260811 19:03:28.075096 31522 tablet_service.cc:1514] Processing DeleteTablet for tablet e669b630ee114961ad830647ab90abc3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:03:07 UTC) from {username='kudu', principal='kudu/127.13.109.62@KRBTEST.COM'} at 127.0.0.1:37076
I20260811 19:03:28.076078 31603 tablet_replica.cc:330] stopping tablet replica
I20260811 19:03:28.076438 31603 raft_consensus.cc:2229] T e669b630ee114961ad830647ab90abc3 P e1052c249dde44e6901a8b7840c6e1bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:03:28.076758 31603 raft_consensus.cc:2258] T e669b630ee114961ad830647ab90abc3 P e1052c249dde44e6901a8b7840c6e1bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:03:28.077999 31603 ts_tablet_manager.cc:1905] T e669b630ee114961ad830647ab90abc3 P e1052c249dde44e6901a8b7840c6e1bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:03:28.081970 31603 ts_tablet_manager.cc:1918] T e669b630ee114961ad830647ab90abc3 P e1052c249dde44e6901a8b7840c6e1bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:03:28.082065 31603 log.cc:1196] T e669b630ee114961ad830647ab90abc3 P e1052c249dde44e6901a8b7840c6e1bb: Deleting WAL directory at /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-1/wal/wals/e669b630ee114961ad830647ab90abc3
I20260811 19:03:28.082367 31603 ts_tablet_manager.cc:1939] T e669b630ee114961ad830647ab90abc3 P e1052c249dde44e6901a8b7840c6e1bb: Deleting consensus metadata
I20260811 19:03:28.084216 31211 catalog_manager.cc:4841] TS e1052c249dde44e6901a8b7840c6e1bb (127.13.109.2:45197): tablet e669b630ee114961ad830647ab90abc3 (table test-table [id=ebc7efa4b6cb46b2ae51391c64a0d047]) successfully deleted
W20260811 19:03:28.177013 31600 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:03:28.177429 31600 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:03:28.177521 31600 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:03:28.182312 31600 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:03:28.182416 31600 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:03:28.182518 31600 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.3
I20260811 19:03:28.188498 31600 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:32993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.3.keytab
--principal=kudu/127.13.109.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.3:40165
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.109.3
--webserver_port=37157
--tserver_master_addrs=127.13.109.62:36875
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31600
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.3
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:03:28.189729 31600 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:03:28.190855 31600 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:03:28.201426 31613 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:03:28.201444 31609 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:03:28.201498 31600 server_base.cc:1034] running on GCE node
W20260811 19:03:28.201444 31611 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:03:28.202119 31600 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:03:28.202775 31600 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:03:28.203941 31600 hybrid_clock.cc:648] HybridClock initialized: now 1786475008203899 us; error 51 us; skew 500 ppm
Aug 11 19:03:28 dist-test-slave-0kls krb5kdc[30569](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475008, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:03:28.207356 31600 init.cc:377] Logged in from keytab as kudu/127.13.109.3@KRBTEST.COM (short username kudu)
I20260811 19:03:28.208384 31600 webserver.cc:460] Webserver started at http://127.13.109.3:37157/ using document root <none> and password file <none>
I20260811 19:03:28.209018 31600 fs_manager.cc:359] Metadata directory not provided
I20260811 19:03:28.209136 31600 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:03:28.213475 31600 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.003s
I20260811 19:03:28.215855 31621 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:03:28.217231 31600 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 19:03:28.217387 31600 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-2/wal
uuid: "a2336f52031842798c0dabb12b7e06e6"
format_stamp: "Formatted at 2026-08-11 19:03:06 on dist-test-slave-0kls"
server_key: "7a614070ce0fa326c55a07c0c28a6cf1"
server_key_iv: "f0757db09afeab1fd0cf5f0af21d1f3e"
server_key_version: "encryptionkey@0"
I20260811 19:03:28.218066 31600 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:03:28.235914 31600 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:03:28.239513 31600 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:03:28.239765 31600 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:03:28.240456 31600 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:03:28.241400 31600 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:03:28.241475 31600 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:03:28.241554 31600 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:03:28.241596 31600 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:03:28.251888 31600 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.3:40165
I20260811 19:03:28.251910 31743 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.3:40165 every 8 connection(s)
I20260811 19:03:28.253059 31600 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
I20260811 19:03:28.258276 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 31600
Aug 11 19:03:28 dist-test-slave-0kls krb5kdc[30569](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475008, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.3@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
I20260811 19:03:28.268359 31749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:28.259050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35164 (local address 127.13.109.62:36875)
0811 19:03:28.259333 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:28.259336 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:28.259456 (+ 120us) server_negotiation.cc:407] Connection header received
0811 19:03:28.259649 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:28.259653 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:28.259705 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:28.259808 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:28.260640 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:28.261423 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:28.262641 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:28.263007 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:28.263108 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:28.264496 (+ 1388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:03:28.264499 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:03:28.264501 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:03:28.264525 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:03:28.266333 (+ 1808us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:28.266855 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:28.266860 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:28.266865 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:28.266958 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:28.267309 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:28.267314 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:28.267317 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:28.267646 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:03:28.267812 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:28.268037 (+ 225us) server_negotiation.cc:299] Negotiation successful
0811 19:03:28.268172 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":219,"thread_start_us":98,"threads_started":1}
I20260811 19:03:28.268927 31439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:28.255267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.3:38387 (local address 127.13.109.62:36875)
0811 19:03:28.255430 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:28.255434 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:28.256665 (+ 1231us) server_negotiation.cc:407] Connection header received
0811 19:03:28.257559 (+ 894us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:28.257562 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:28.257627 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:28.257710 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:28.259273 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:28.260055 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:28.261728 (+ 1673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:28.262123 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:28.262231 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:28.264978 (+ 2747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:03:28.264981 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:03:28.264984 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:03:28.265030 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:03:28.266517 (+ 1487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:28.267142 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:28.267146 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:28.267148 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:28.267237 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:28.267627 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:28.267629 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:28.267630 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:28.267858 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:03:28.267967 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:28.268669 (+ 702us) server_negotiation.cc:299] Negotiation successful
0811 19:03:28.268789 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 19:03:28.270023 31746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:28.255724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:36875 (local address 127.13.109.3:38387)
0811 19:03:28.256468 (+ 744us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:28.256515 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 19:03:28.257335 (+ 820us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:03:28.257983 (+ 648us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:03:28.257991 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:03:28.258382 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:03:28.259058 (+ 676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:03:28.259076 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:03:28.260190 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:03:28.260193 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:03:28.261583 (+ 1390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:03:28.261593 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:03:28.262288 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:03:28.262291 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:03:28.262494 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:28.263161 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:03:28.263186 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:03:28.264798 (+ 1612us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:03:28.266627 (+ 1829us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:03:28.266633 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:03:28.266648 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:03:28.267033 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:03:28.267334 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:03:28.267337 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:03:28.267339 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:03:28.267522 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:03:28.267997 (+ 475us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:03:28.268004 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:03:28.268368 (+ 364us) client_negotiation.cc:769] Sending connection context
0811 19:03:28.268602 (+ 234us) client_negotiation.cc:240] Negotiation successful
0811 19:03:28.268978 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":132,"threads_started":1}
I20260811 19:03:28.271245 31744 heartbeater.cc:347] Connected to a master server at 127.13.109.62:36875
I20260811 19:03:28.271515 31744 heartbeater.cc:464] Registering TS with master...
I20260811 19:03:28.272053 31744 heartbeater.cc:511] Master 127.13.109.62:36875 requested a full tablet report, sending...
I20260811 19:03:28.273108 31228 ts_manager.cc:194] Registered new tserver with Master: a2336f52031842798c0dabb12b7e06e6 (127.13.109.3:40165)
I20260811 19:03:28.273695 31228 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.109.3@KRBTEST.COM'} at 127.13.109.3:38387
I20260811 19:03:28.282742 13748 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 19:03:28.287331 31439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:28.283427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35180 (local address 127.13.109.62:36875)
0811 19:03:28.283610 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:28.283613 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:28.283756 (+ 143us) server_negotiation.cc:407] Connection header received
0811 19:03:28.283906 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:28.283909 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:28.284029 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:28.284129 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:03:28.284381 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:28.284925 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:28.285753 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:28.286242 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:28.286352 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:28.286645 (+ 293us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:03:28.286824 (+ 179us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0811 19:03:28.287166 (+ 342us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:35180: token expired
Metrics: {"server-negotiator.queue_time_us":65}
W20260811 19:03:28.287437 31439 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:35180: token expired
W20260811 19:03:28.287716 31137 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.13.109.62:36875)
I20260811 19:03:28.295367 31439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:28.288075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35192 (local address 127.13.109.62:36875)
0811 19:03:28.288224 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:28.288227 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:28.288239 (+ 12us) server_negotiation.cc:407] Connection header received
0811 19:03:28.288326 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:28.288328 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:28.288367 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:28.288429 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:28.289091 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:28.289564 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:28.290515 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:28.290733 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:28.290813 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:28.292378 (+ 1565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:03:28.292381 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:03:28.292384 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:03:28.292406 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:03:28.293778 (+ 1372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:28.294244 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:28.294247 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:28.294248 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:28.294322 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:28.294554 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:28.294556 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:28.294558 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:28.294741 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:03:28.294841 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:28.295139 (+ 298us) server_negotiation.cc:299] Negotiation successful
0811 19:03:28.295239 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 19:03:28.298424 31228 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35192:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 19:03:28.301906 31228 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 19:03:28.310269 31366 tablet_service.cc:1467] Processing CreateTablet for tablet e3a9d0813da9442b87967955d0f6b2e9 (DEFAULT_TABLE table=test-table [id=48e3941ce81d49c6a490ac2ffba4e86d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:03:28.310227 31522 tablet_service.cc:1467] Processing CreateTablet for tablet e3a9d0813da9442b87967955d0f6b2e9 (DEFAULT_TABLE table=test-table [id=48e3941ce81d49c6a490ac2ffba4e86d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:03:28.310940 31522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3a9d0813da9442b87967955d0f6b2e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:03:28.310940 31366 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3a9d0813da9442b87967955d0f6b2e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:03:28.313740 31321 tablet_bootstrap.cc:492] T e3a9d0813da9442b87967955d0f6b2e9 P 0b24c35b827f4b0badf0b66aec15cae9: Bootstrap starting.
I20260811 19:03:28.313760 31474 tablet_bootstrap.cc:492] T e3a9d0813da9442b87967955d0f6b2e9 P e1052c249dde44e6901a8b7840c6e1bb: Bootstrap starting.
I20260811 19:03:28.314817 31474 tablet_bootstrap.cc:654] T e3a9d0813da9442b87967955d0f6b2e9 P e1052c249dde44e6901a8b7840c6e1bb: Neither blocks nor log segments found. Creating new log.
I20260811 19:03:28.314841 31321 tablet_bootstrap.cc:654] T e3a9d0813da9442b87967955d0f6b2e9 P 0b24c35b827f4b0badf0b66aec15cae9: Neither blocks nor log segments found. Creating new log.
I20260811 19:03:28.315886 31445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:28.309423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.3:40165 (local address 127.0.0.1:48418)
0811 19:03:28.309572 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:28.309589 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 19:03:28.309675 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:03:28.310789 (+ 1114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:03:28.310793 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:03:28.310813 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:03:28.310923 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:03:28.310931 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:03:28.312529 (+ 1598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:03:28.312533 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:03:28.313879 (+ 1346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:03:28.313893 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:03:28.315396 (+ 1503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:03:28.315402 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:03:28.315505 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:28.315578 (+ 73us) client_negotiation.cc:769] Sending connection context
0811 19:03:28.315640 (+ 62us) client_negotiation.cc:240] Negotiation successful
0811 19:03:28.315730 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260811 19:03:28.316463 31321 tablet_bootstrap.cc:492] T e3a9d0813da9442b87967955d0f6b2e9 P 0b24c35b827f4b0badf0b66aec15cae9: No bootstrap required, opened a new log
I20260811 19:03:28.316560 31321 ts_tablet_manager.cc:1397] T e3a9d0813da9442b87967955d0f6b2e9 P 0b24c35b827f4b0badf0b66aec15cae9: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260811 19:03:28.316818 31754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:28.309810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48418 (local address 127.13.109.3:40165)
0811 19:03:28.310225 (+ 415us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:28.310240 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:28.310268 (+ 28us) server_negotiation.cc:407] Connection header received
0811 19:03:28.310357 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:28.310363 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:28.310601 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:28.310792 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:03:28.311212 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:28.312380 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:28.314144 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:28.315215 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:28.316124 (+ 909us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:28.316303 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:28.316363 (+ 60us) server_negotiation.cc:299] Negotiation successful
0811 19:03:28.316477 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":155,"threads_started":1}
I20260811 19:03:28.317076 31321 raft_consensus.cc:348] T e3a9d0813da9442b87967955d0f6b2e9 P 0b24c35b827f4b0badf0b66aec15cae9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2336f52031842798c0dabb12b7e06e6" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 40165 } } peers { permanent_uuid: "e1052c249dde44e6901a8b7840c6e1bb" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 45197 } } peers { permanent_uuid: "0b24c35b827f4b0badf0b66aec15cae9" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 36839 } }
I20260811 19:03:28.317216 31321 raft_consensus.cc:374] T e3a9d0813da9442b87967955d0f6b2e9 P 0b24c35b827f4b0badf0b66aec15cae9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:03:28.317258 31321 raft_consensus.cc:729] T e3a9d0813da9442b87967955d0f6b2e9 P 0b24c35b827f4b0badf0b66aec15cae9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b24c35b827f4b0badf0b66aec15cae9, State: Initialized, Role: FOLLOWER
I20260811 19:03:28.317389 31321 consensus_queue.cc:260] T e3a9d0813da9442b87967955d0f6b2e9 P 0b24c35b827f4b0badf0b66aec15cae9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2336f52031842798c0dabb12b7e06e6" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 40165 } } peers { permanent_uuid: "e1052c249dde44e6901a8b7840c6e1bb" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 45197 } } peers { permanent_uuid: "0b24c35b827f4b0badf0b66aec15cae9" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 36839 } }
I20260811 19:03:28.317641 31321 ts_tablet_manager.cc:1428] T e3a9d0813da9442b87967955d0f6b2e9 P 0b24c35b827f4b0badf0b66aec15cae9: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 19:03:28.318866 31474 tablet_bootstrap.cc:492] T e3a9d0813da9442b87967955d0f6b2e9 P e1052c249dde44e6901a8b7840c6e1bb: No bootstrap required, opened a new log
I20260811 19:03:28.318992 31474 ts_tablet_manager.cc:1397] T e3a9d0813da9442b87967955d0f6b2e9 P e1052c249dde44e6901a8b7840c6e1bb: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20260811 19:03:28.319518 31474 raft_consensus.cc:348] T e3a9d0813da9442b87967955d0f6b2e9 P e1052c249dde44e6901a8b7840c6e1bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2336f52031842798c0dabb12b7e06e6" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 40165 } } peers { permanent_uuid: "e1052c249dde44e6901a8b7840c6e1bb" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 45197 } } peers { permanent_uuid: "0b24c35b827f4b0badf0b66aec15cae9" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 36839 } }
I20260811 19:03:28.319655 31474 raft_consensus.cc:374] T e3a9d0813da9442b87967955d0f6b2e9 P e1052c249dde44e6901a8b7840c6e1bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:03:28.319708 31474 raft_consensus.cc:729] T e3a9d0813da9442b87967955d0f6b2e9 P e1052c249dde44e6901a8b7840c6e1bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1052c249dde44e6901a8b7840c6e1bb, State: Initialized, Role: FOLLOWER
I20260811 19:03:28.319885 31474 consensus_queue.cc:260] T e3a9d0813da9442b87967955d0f6b2e9 P e1052c249dde44e6901a8b7840c6e1bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2336f52031842798c0dabb12b7e06e6" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 40165 } } peers { permanent_uuid: "e1052c249dde44e6901a8b7840c6e1bb" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 45197 } } peers { permanent_uuid: "0b24c35b827f4b0badf0b66aec15cae9" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 36839 } }
I20260811 19:03:28.320187 31474 ts_tablet_manager.cc:1428] T e3a9d0813da9442b87967955d0f6b2e9 P e1052c249dde44e6901a8b7840c6e1bb: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 19:03:28.320463 31675 tablet_service.cc:1467] Processing CreateTablet for tablet e3a9d0813da9442b87967955d0f6b2e9 (DEFAULT_TABLE table=test-table [id=48e3941ce81d49c6a490ac2ffba4e86d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:03:28.321760 31675 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3a9d0813da9442b87967955d0f6b2e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:03:28.330283 31758 tablet_bootstrap.cc:492] T e3a9d0813da9442b87967955d0f6b2e9 P a2336f52031842798c0dabb12b7e06e6: Bootstrap starting.
I20260811 19:03:28.332643 31758 tablet_bootstrap.cc:654] T e3a9d0813da9442b87967955d0f6b2e9 P a2336f52031842798c0dabb12b7e06e6: Neither blocks nor log segments found. Creating new log.
I20260811 19:03:28.333866 31758 log.cc:824] T e3a9d0813da9442b87967955d0f6b2e9 P a2336f52031842798c0dabb12b7e06e6: Log is configured to *not* fsync() on all Append() calls
I20260811 19:03:28.335891 31758 tablet_bootstrap.cc:492] T e3a9d0813da9442b87967955d0f6b2e9 P a2336f52031842798c0dabb12b7e06e6: No bootstrap required, opened a new log
I20260811 19:03:28.336120 31758 ts_tablet_manager.cc:1397] T e3a9d0813da9442b87967955d0f6b2e9 P a2336f52031842798c0dabb12b7e06e6: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 19:03:28.336784 31444 raft_consensus.cc:482] T e3a9d0813da9442b87967955d0f6b2e9 P 0b24c35b827f4b0badf0b66aec15cae9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:03:28.337002 31444 raft_consensus.cc:504] T e3a9d0813da9442b87967955d0f6b2e9 P 0b24c35b827f4b0badf0b66aec15cae9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2336f52031842798c0dabb12b7e06e6" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 40165 } } peers { permanent_uuid: "e1052c249dde44e6901a8b7840c6e1bb" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 45197 } } peers { permanent_uuid: "0b24c35b827f4b0badf0b66aec15cae9" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 36839 } }
I20260811 19:03:28.338366 31444 leader_election.cc:288] T e3a9d0813da9442b87967955d0f6b2e9 P 0b24c35b827f4b0badf0b66aec15cae9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2336f52031842798c0dabb12b7e06e6 (127.13.109.3:40165), e1052c249dde44e6901a8b7840c6e1bb (127.13.109.2:45197)
I20260811 19:03:28.340306 31758 raft_consensus.cc:348] T e3a9d0813da9442b87967955d0f6b2e9 P a2336f52031842798c0dabb12b7e06e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2336f52031842798c0dabb12b7e06e6" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 40165 } } peers { permanent_uuid: "e1052c249dde44e6901a8b7840c6e1bb" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 45197 } } peers { permanent_uuid: "0b24c35b827f4b0badf0b66aec15cae9" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 36839 } }
I20260811 19:03:28.340641 31758 raft_consensus.cc:374] T e3a9d0813da9442b87967955d0f6b2e9 P a2336f52031842798c0dabb12b7e06e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:03:28.340703 31758 raft_consensus.cc:729] T e3a9d0813da9442b87967955d0f6b2e9 P a2336f52031842798c0dabb12b7e06e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2336f52031842798c0dabb12b7e06e6, State: Initialized, Role: FOLLOWER
I20260811 19:03:28.341249 31758 consensus_queue.cc:260] T e3a9d0813da9442b87967955d0f6b2e9 P a2336f52031842798c0dabb12b7e06e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2336f52031842798c0dabb12b7e06e6" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 40165 } } peers { permanent_uuid: "e1052c249dde44e6901a8b7840c6e1bb" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 45197 } } peers { permanent_uuid: "0b24c35b827f4b0badf0b66aec15cae9" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 36839 } }
I20260811 19:03:28.342052 31744 heartbeater.cc:503] Master 127.13.109.62:36875 was elected leader, sending a full tablet report...
I20260811 19:03:28.342505 31758 ts_tablet_manager.cc:1428] T e3a9d0813da9442b87967955d0f6b2e9 P a2336f52031842798c0dabb12b7e06e6: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 19:03:28.343673 31754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:28.339138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.1:48107 (local address 127.13.109.3:40165)
0811 19:03:28.339699 (+ 561us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:28.339703 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:28.339716 (+ 13us) server_negotiation.cc:407] Connection header received
0811 19:03:28.339786 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:28.339792 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:28.339840 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:28.339916 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:03:28.340185 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:28.340717 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:28.342300 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:28.342876 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:28.342979 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:28.343032 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:28.343447 (+ 415us) server_negotiation.cc:299] Negotiation successful
0811 19:03:28.343535 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":477}
I20260811 19:03:28.343681 31762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:28.338547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.3:40165 (local address 127.13.109.1:48107)
0811 19:03:28.338973 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:28.338991 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 19:03:28.339107 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:03:28.339934 (+ 827us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:03:28.339937 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:03:28.339953 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:03:28.340069 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:03:28.340075 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:03:28.340860 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:03:28.340863 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:03:28.342152 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:03:28.342164 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:03:28.343160 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:03:28.343164 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:03:28.343277 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:28.343331 (+ 54us) client_negotiation.cc:769] Sending connection context
0811 19:03:28.343411 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 19:03:28.343505 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"mutex_wait_us":51,"thread_start_us":108,"threads_started":1}
I20260811 19:03:28.343860 31602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:28.338987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.1:41019 (local address 127.13.109.2:45197)
0811 19:03:28.339292 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:28.339296 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:28.339310 (+ 14us) server_negotiation.cc:407] Connection header received
0811 19:03:28.339368 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:28.339370 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:28.339419 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:28.339522 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:03:28.339860 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:28.340664 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:28.342309 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:28.342972 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:28.343056 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:28.343225 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:28.343617 (+ 392us) server_negotiation.cc:299] Negotiation successful
0811 19:03:28.343719 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204}
I20260811 19:03:28.344277 31761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:28.338685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.2:45197 (local address 127.13.109.1:41019)
0811 19:03:28.339097 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:28.339112 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 19:03:28.339206 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:03:28.339535 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:03:28.339538 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:03:28.339555 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:03:28.339697 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:03:28.339704 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:03:28.340819 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:03:28.340822 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:03:28.342147 (+ 1325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:03:28.342161 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:03:28.343355 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:03:28.343358 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:03:28.343475 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:28.343498 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 19:03:28.344008 (+ 510us) client_negotiation.cc:240] Negotiation successful
0811 19:03:28.344109 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"mutex_wait_us":71,"thread_start_us":93,"threads_started":1}
I20260811 19:03:28.344408 31696 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3a9d0813da9442b87967955d0f6b2e9" candidate_uuid: "0b24c35b827f4b0badf0b66aec15cae9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2336f52031842798c0dabb12b7e06e6" is_pre_election: true
I20260811 19:03:28.344695 31696 raft_consensus.cc:2454] T e3a9d0813da9442b87967955d0f6b2e9 P a2336f52031842798c0dabb12b7e06e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b24c35b827f4b0badf0b66aec15cae9 in term 0.
I20260811 19:03:28.345012 31544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3a9d0813da9442b87967955d0f6b2e9" candidate_uuid: "0b24c35b827f4b0badf0b66aec15cae9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1052c249dde44e6901a8b7840c6e1bb" is_pre_election: true
I20260811 19:03:28.345244 31313 leader_election.cc:302] T e3a9d0813da9442b87967955d0f6b2e9 P 0b24c35b827f4b0badf0b66aec15cae9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b24c35b827f4b0badf0b66aec15cae9, a2336f52031842798c0dabb12b7e06e6; no voters:
I20260811 19:03:28.345315 31544 raft_consensus.cc:2454] T e3a9d0813da9442b87967955d0f6b2e9 P e1052c249dde44e6901a8b7840c6e1bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b24c35b827f4b0badf0b66aec15cae9 in term 0.
I20260811 19:03:28.345551 31444 raft_consensus.cc:2781] T e3a9d0813da9442b87967955d0f6b2e9 P 0b24c35b827f4b0badf0b66aec15cae9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 19:03:28.345659 31444 raft_consensus.cc:482] T e3a9d0813da9442b87967955d0f6b2e9 P 0b24c35b827f4b0badf0b66aec15cae9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 19:03:28.345718 31444 raft_consensus.cc:3037] T e3a9d0813da9442b87967955d0f6b2e9 P 0b24c35b827f4b0badf0b66aec15cae9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:03:28.347267 31444 raft_consensus.cc:504] T e3a9d0813da9442b87967955d0f6b2e9 P 0b24c35b827f4b0badf0b66aec15cae9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2336f52031842798c0dabb12b7e06e6" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 40165 } } peers { permanent_uuid: "e1052c249dde44e6901a8b7840c6e1bb" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 45197 } } peers { permanent_uuid: "0b24c35b827f4b0badf0b66aec15cae9" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 36839 } }
I20260811 19:03:28.347663 31444 leader_election.cc:288] T e3a9d0813da9442b87967955d0f6b2e9 P 0b24c35b827f4b0badf0b66aec15cae9 [CANDIDATE]: Term 1 election: Requested vote from peers a2336f52031842798c0dabb12b7e06e6 (127.13.109.3:40165), e1052c249dde44e6901a8b7840c6e1bb (127.13.109.2:45197)
I20260811 19:03:28.348061 31544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3a9d0813da9442b87967955d0f6b2e9" candidate_uuid: "0b24c35b827f4b0badf0b66aec15cae9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1052c249dde44e6901a8b7840c6e1bb"
I20260811 19:03:28.348196 31544 raft_consensus.cc:3037] T e3a9d0813da9442b87967955d0f6b2e9 P e1052c249dde44e6901a8b7840c6e1bb [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:03:28.349383 31544 raft_consensus.cc:2454] T e3a9d0813da9442b87967955d0f6b2e9 P e1052c249dde44e6901a8b7840c6e1bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b24c35b827f4b0badf0b66aec15cae9 in term 1.
I20260811 19:03:28.349694 31696 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3a9d0813da9442b87967955d0f6b2e9" candidate_uuid: "0b24c35b827f4b0badf0b66aec15cae9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2336f52031842798c0dabb12b7e06e6"
I20260811 19:03:28.349735 31314 leader_election.cc:302] T e3a9d0813da9442b87967955d0f6b2e9 P 0b24c35b827f4b0badf0b66aec15cae9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b24c35b827f4b0badf0b66aec15cae9, e1052c249dde44e6901a8b7840c6e1bb; no voters:
I20260811 19:03:28.349879 31696 raft_consensus.cc:3037] T e3a9d0813da9442b87967955d0f6b2e9 P a2336f52031842798c0dabb12b7e06e6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:03:28.349989 31444 raft_consensus.cc:2781] T e3a9d0813da9442b87967955d0f6b2e9 P 0b24c35b827f4b0badf0b66aec15cae9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:03:28.350263 31444 raft_consensus.cc:686] T e3a9d0813da9442b87967955d0f6b2e9 P 0b24c35b827f4b0badf0b66aec15cae9 [term 1 LEADER]: Becoming Leader. State: Replica: 0b24c35b827f4b0badf0b66aec15cae9, State: Running, Role: LEADER
I20260811 19:03:28.350591 31444 consensus_queue.cc:237] T e3a9d0813da9442b87967955d0f6b2e9 P 0b24c35b827f4b0badf0b66aec15cae9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2336f52031842798c0dabb12b7e06e6" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 40165 } } peers { permanent_uuid: "e1052c249dde44e6901a8b7840c6e1bb" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 45197 } } peers { permanent_uuid: "0b24c35b827f4b0badf0b66aec15cae9" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 36839 } }
I20260811 19:03:28.351330 31696 raft_consensus.cc:2454] T e3a9d0813da9442b87967955d0f6b2e9 P a2336f52031842798c0dabb12b7e06e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b24c35b827f4b0badf0b66aec15cae9 in term 1.
I20260811 19:03:28.354270 31228 catalog_manager.cc:5495] T e3a9d0813da9442b87967955d0f6b2e9 P 0b24c35b827f4b0badf0b66aec15cae9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0b24c35b827f4b0badf0b66aec15cae9 (127.13.109.1). New cstate: current_term: 1 leader_uuid: "0b24c35b827f4b0badf0b66aec15cae9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2336f52031842798c0dabb12b7e06e6" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 40165 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e1052c249dde44e6901a8b7840c6e1bb" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 45197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b24c35b827f4b0badf0b66aec15cae9" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 36839 } health_report { overall_health: HEALTHY } } }
I20260811 19:03:28.375002 31768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:28.371086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40854 (local address 127.13.109.1:36839)
0811 19:03:28.371386 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:28.371391 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:28.371413 (+ 22us) server_negotiation.cc:407] Connection header received
0811 19:03:28.371496 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:28.371498 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:28.371554 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:28.371692 (+ 138us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:03:28.371957 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:28.372491 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:28.373379 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:28.373621 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:28.373743 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:28.374064 (+ 321us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:03:28.374561 (+ 497us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:03:28.374704 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:28.374755 (+ 51us) server_negotiation.cc:299] Negotiation successful
0811 19:03:28.374821 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":84,"threads_started":1}
I20260811 19:03:28.379679 31696 raft_consensus.cc:1261] T e3a9d0813da9442b87967955d0f6b2e9 P a2336f52031842798c0dabb12b7e06e6 [term 1 FOLLOWER]: Refusing update from remote peer 0b24c35b827f4b0badf0b66aec15cae9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:03:28.379678 31544 raft_consensus.cc:1261] T e3a9d0813da9442b87967955d0f6b2e9 P e1052c249dde44e6901a8b7840c6e1bb [term 1 FOLLOWER]: Refusing update from remote peer 0b24c35b827f4b0badf0b66aec15cae9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:03:28.380637 31444 consensus_queue.cc:1035] T e3a9d0813da9442b87967955d0f6b2e9 P 0b24c35b827f4b0badf0b66aec15cae9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1052c249dde44e6901a8b7840c6e1bb" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 45197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:03:28.380906 31764 consensus_queue.cc:1035] T e3a9d0813da9442b87967955d0f6b2e9 P 0b24c35b827f4b0badf0b66aec15cae9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2336f52031842798c0dabb12b7e06e6" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 40165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:03:28.392021 31770 mvcc.cc:205] Tried to move back new op lower bound from 7317401634314645504 to 7317401634207555584. Current Snapshot: MvccSnapshot[applied={T|T < 7317401634314645504 or (T in {7317401634314645504})}]
I20260811 19:03:28.398326 31227 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35192:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:03:28.398548 31227 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35192:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:03:28.401304 31227 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7d3cc91d778246cdb59c93831c15d546: Sending DeleteTablet for 3 replicas of tablet e3a9d0813da9442b87967955d0f6b2e9
I20260811 19:03:28.402225 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 31271
I20260811 19:03:28.402227 31522 tablet_service.cc:1514] Processing DeleteTablet for tablet e3a9d0813da9442b87967955d0f6b2e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:03:28 UTC) from {username='kudu', principal='kudu/127.13.109.62@KRBTEST.COM'} at 127.0.0.1:37076
I20260811 19:03:28.402364 31366 tablet_service.cc:1514] Processing DeleteTablet for tablet e3a9d0813da9442b87967955d0f6b2e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:03:28 UTC) from {username='kudu', principal='kudu/127.13.109.62@KRBTEST.COM'} at 127.0.0.1:40842
I20260811 19:03:28.402447 31603 tablet_replica.cc:330] T e3a9d0813da9442b87967955d0f6b2e9 P e1052c249dde44e6901a8b7840c6e1bb: stopping tablet replica
I20260811 19:03:28.402588 31603 raft_consensus.cc:2229] T e3a9d0813da9442b87967955d0f6b2e9 P e1052c249dde44e6901a8b7840c6e1bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:03:28.402729 31603 raft_consensus.cc:2258] T e3a9d0813da9442b87967955d0f6b2e9 P e1052c249dde44e6901a8b7840c6e1bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:03:28.402941 31675 tablet_service.cc:1514] Processing DeleteTablet for tablet e3a9d0813da9442b87967955d0f6b2e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:03:28 UTC) from {username='kudu', principal='kudu/127.13.109.62@KRBTEST.COM'} at 127.0.0.1:48418
I20260811 19:03:28.404248 31778 tablet_replica.cc:330] T e3a9d0813da9442b87967955d0f6b2e9 P a2336f52031842798c0dabb12b7e06e6: stopping tablet replica
I20260811 19:03:28.404765 31778 raft_consensus.cc:2229] T e3a9d0813da9442b87967955d0f6b2e9 P a2336f52031842798c0dabb12b7e06e6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:03:28.405179 31778 raft_consensus.cc:2258] T e3a9d0813da9442b87967955d0f6b2e9 P a2336f52031842798c0dabb12b7e06e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:03:28.405771 31603 ts_tablet_manager.cc:1905] T e3a9d0813da9442b87967955d0f6b2e9 P e1052c249dde44e6901a8b7840c6e1bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:03:28.407323 31778 ts_tablet_manager.cc:1905] T e3a9d0813da9442b87967955d0f6b2e9 P a2336f52031842798c0dabb12b7e06e6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:03:28.409163 31603 ts_tablet_manager.cc:1918] T e3a9d0813da9442b87967955d0f6b2e9 P e1052c249dde44e6901a8b7840c6e1bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:03:28.409286 31603 log.cc:1196] T e3a9d0813da9442b87967955d0f6b2e9 P e1052c249dde44e6901a8b7840c6e1bb: Deleting WAL directory at /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-1/wal/wals/e3a9d0813da9442b87967955d0f6b2e9
I20260811 19:03:28.409577 31603 ts_tablet_manager.cc:1939] T e3a9d0813da9442b87967955d0f6b2e9 P e1052c249dde44e6901a8b7840c6e1bb: Deleting consensus metadata
I20260811 19:03:28.410650 31211 catalog_manager.cc:4841] TS e1052c249dde44e6901a8b7840c6e1bb (127.13.109.2:45197): tablet e3a9d0813da9442b87967955d0f6b2e9 (table test-table [id=48e3941ce81d49c6a490ac2ffba4e86d]) successfully deleted
I20260811 19:03:28.411403 31778 ts_tablet_manager.cc:1918] T e3a9d0813da9442b87967955d0f6b2e9 P a2336f52031842798c0dabb12b7e06e6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:03:28.411522 31778 log.cc:1196] T e3a9d0813da9442b87967955d0f6b2e9 P a2336f52031842798c0dabb12b7e06e6: Deleting WAL directory at /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-2/wal/wals/e3a9d0813da9442b87967955d0f6b2e9
I20260811 19:03:28.411979 31778 ts_tablet_manager.cc:1939] T e3a9d0813da9442b87967955d0f6b2e9 P a2336f52031842798c0dabb12b7e06e6: Deleting consensus metadata
I20260811 19:03:28.413563 31210 catalog_manager.cc:4841] TS a2336f52031842798c0dabb12b7e06e6 (127.13.109.3:40165): tablet e3a9d0813da9442b87967955d0f6b2e9 (table test-table [id=48e3941ce81d49c6a490ac2ffba4e86d]) successfully deleted
W20260811 19:03:28.414192 31210 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.109.1:36839 (error 108)
I20260811 19:03:28.415371 31445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:28.415032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.1:36839 (local address 127.0.0.1:40856)
0811 19:03:28.415173 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:28.415256 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.1:36839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260811 19:03:28.415584 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 31441
W20260811 19:03:28.415603 31210 catalog_manager.cc:4584] TS 0b24c35b827f4b0badf0b66aec15cae9 (127.13.109.1:36839): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e3a9d0813da9442b87967955d0f6b2e9: Network error: Client connection negotiation failed: client connection to 127.13.109.1:36839: connect: Connection refused (error 111)
I20260811 19:03:28.428227 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 31600
I20260811 19:03:28.441489 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 31191
I20260811 19:03:48.454341 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.62:36875
--webserver_interface=127.13.109.62
--webserver_port=42837
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab
--principal=kudu/127.13.109.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:32993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.109.62:36875
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:03:48.582927 31780 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:03:48.583230 31780 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:03:48.583315 31780 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:03:48.587643 31780 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 19:03:48.587725 31780 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 19:03:48.587742 31780 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 19:03:48.587757 31780 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:03:48.587822 31780 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:03:48.587858 31780 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:03:48.587894 31780 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:03:48.587932 31780 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:03:48.593993 31780 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:32993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.13.109.62:36875
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab
--principal=kudu/127.13.109.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.62:36875
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.109.62
--webserver_port=42837
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31780
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:03:48.595199 31780 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:03:48.596295 31780 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:03:48.605744 31790 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:03:48.605777 31786 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:03:48.605837 31780 server_base.cc:1034] running on GCE node
W20260811 19:03:48.605955 31787 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:03:48.606724 31780 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:03:48.607918 31780 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:03:48.609091 31780 hybrid_clock.cc:648] HybridClock initialized: now 1786475028609062 us; error 43 us; skew 500 ppm
Aug 11 19:03:48 dist-test-slave-0kls krb5kdc[30569](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475028, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:03:48.612247 31780 init.cc:377] Logged in from keytab as kudu/127.13.109.62@KRBTEST.COM (short username kudu)
I20260811 19:03:48.613327 31780 webserver.cc:460] Webserver started at http://127.13.109.62:42837/ using document root <none> and password file <none>
I20260811 19:03:48.614032 31780 fs_manager.cc:359] Metadata directory not provided
I20260811 19:03:48.614131 31780 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:03:48.618434 31780 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 19:03:48.620920 31797 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:03:48.622673 31780 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 19:03:48.622839 31780 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/master-0/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/master-0/wal
uuid: "7d3cc91d778246cdb59c93831c15d546"
format_stamp: "Formatted at 2026-08-11 19:03:05 on dist-test-slave-0kls"
server_key: "e47d5437894a1d90f80f2212a32b4840"
server_key_iv: "96c4a8f843d3b7c7b8f8538711c6ed69"
server_key_version: "encryptionkey@0"
I20260811 19:03:48.623478 31780 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:03:48.635612 31780 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:03:48.639120 31780 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:03:48.639367 31780 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:03:48.647294 31780 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.62:36875
I20260811 19:03:48.647619 31856 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.62:36875 every 8 connection(s)
I20260811 19:03:48.648486 31780 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
I20260811 19:03:48.653223 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 31780
I20260811 19:03:48.653435 31857 sys_catalog.cc:263] Verifying existing consensus state
I20260811 19:03:48.653839 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.1:36839
--local_ip_for_outbound_sockets=127.13.109.1
--tserver_master_addrs=127.13.109.62:36875
--webserver_port=33679
--webserver_interface=127.13.109.1
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.1.keytab
--principal=kudu/127.13.109.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:32993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 19:03:48.656239 31857 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d3cc91d778246cdb59c93831c15d546: Bootstrap starting.
I20260811 19:03:48.675493 31857 log.cc:824] T 00000000000000000000000000000000 P 7d3cc91d778246cdb59c93831c15d546: Log is configured to *not* fsync() on all Append() calls
I20260811 19:03:48.688649 31857 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d3cc91d778246cdb59c93831c15d546: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 19:03:48.689160 31857 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d3cc91d778246cdb59c93831c15d546: Bootstrap complete.
I20260811 19:03:48.693596 31857 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7d3cc91d778246cdb59c93831c15d546 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d3cc91d778246cdb59c93831c15d546" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 36875 } }
I20260811 19:03:48.694150 31857 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7d3cc91d778246cdb59c93831c15d546 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d3cc91d778246cdb59c93831c15d546, State: Initialized, Role: FOLLOWER
I20260811 19:03:48.694773 31857 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7d3cc91d778246cdb59c93831c15d546 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d3cc91d778246cdb59c93831c15d546" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 36875 } }
I20260811 19:03:48.694904 31857 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7d3cc91d778246cdb59c93831c15d546 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:03:48.694998 31857 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7d3cc91d778246cdb59c93831c15d546 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:03:48.695102 31857 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7d3cc91d778246cdb59c93831c15d546 [term 2 FOLLOWER]: Advancing to term 3
I20260811 19:03:48.697350 31857 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7d3cc91d778246cdb59c93831c15d546 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d3cc91d778246cdb59c93831c15d546" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 36875 } }
I20260811 19:03:48.697758 31857 leader_election.cc:302] T 00000000000000000000000000000000 P 7d3cc91d778246cdb59c93831c15d546 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7d3cc91d778246cdb59c93831c15d546; no voters:
I20260811 19:03:48.698227 31857 leader_election.cc:288] T 00000000000000000000000000000000 P 7d3cc91d778246cdb59c93831c15d546 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 19:03:48.698329 31861 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7d3cc91d778246cdb59c93831c15d546 [term 3 FOLLOWER]: Leader election won for term 3
I20260811 19:03:48.698549 31861 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7d3cc91d778246cdb59c93831c15d546 [term 3 LEADER]: Becoming Leader. State: Replica: 7d3cc91d778246cdb59c93831c15d546, State: Running, Role: LEADER
I20260811 19:03:48.698956 31861 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7d3cc91d778246cdb59c93831c15d546 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d3cc91d778246cdb59c93831c15d546" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 36875 } }
I20260811 19:03:48.699668 31857 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7d3cc91d778246cdb59c93831c15d546 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:03:48.700625 31863 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d3cc91d778246cdb59c93831c15d546 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7d3cc91d778246cdb59c93831c15d546. Latest consensus state: current_term: 3 leader_uuid: "7d3cc91d778246cdb59c93831c15d546" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d3cc91d778246cdb59c93831c15d546" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 36875 } } }
I20260811 19:03:48.700767 31863 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d3cc91d778246cdb59c93831c15d546 [sys.catalog]: This master's current role is: LEADER
I20260811 19:03:48.701061 31862 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d3cc91d778246cdb59c93831c15d546 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "7d3cc91d778246cdb59c93831c15d546" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d3cc91d778246cdb59c93831c15d546" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 36875 } } }
I20260811 19:03:48.701157 31862 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d3cc91d778246cdb59c93831c15d546 [sys.catalog]: This master's current role is: LEADER
I20260811 19:03:48.705103 31877 catalog_manager.cc:1259] Loaded cluster ID: aa1f3e5656714d4fb6a72e293346f4f4
I20260811 19:03:48.705199 31877 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7d3cc91d778246cdb59c93831c15d546: loading cluster ID for follower catalog manager: success
I20260811 19:03:48.707161 31877 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7d3cc91d778246cdb59c93831c15d546: acquiring CA information for follower catalog manager: success
F20260811 19:03:48.708204 31877 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f51070dd9ad google::LogMessage::Fail() at ??:0
@ 0x7f51070e088a google::LogMessage::SendToLog() at ??:0
@ 0x7f51070dd4de google::LogMessage::Flush() at ??:0
@ 0x7f51070dea19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f5109da118d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f5111a42989 kudu::master::CatalogManager::LoadTspkEntries() at ??:0
@ 0x7f5111a1f023 kudu::master::CatalogManager::PrepareFollowerTokenVerifier() at ??:0
@ 0x7f5111a1f54b kudu::master::CatalogManager::PrepareFollower() at ??:0
@ 0x7f5111a18652 kudu::master::CatalogManagerBgTasks::Run() at ??:0
@ 0x7f5111a179e9 _ZZN4kudu6master21CatalogManagerBgTasks4InitEvENKUlvE_clEv at ??:0
@ 0x7f5111a5e541 _ZNSt17_Function_handlerIFvvEZN4kudu6master21CatalogManagerBgTasks4InitEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f5109320512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f510a9646db start_thread at ??:0
@ 0x7f5105bdb61f clone at ??:0
W20260811 19:03:48.797089 31859 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:03:48.797417 31859 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:03:48.797492 31859 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:03:48.801877 31859 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:03:48.801970 31859 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:03:48.802064 31859 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.1
I20260811 19:03:48.807802 31859 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:32993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.1.keytab
--principal=kudu/127.13.109.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.1:36839
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.109.1
--webserver_port=33679
--tserver_master_addrs=127.13.109.62:36875
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31859
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.1
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:03:48.809002 31859 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:03:48.810137 31859 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:03:48.820187 31904 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:03:48.820187 31903 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:03:48.820369 31906 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:03:48.821066 31859 server_base.cc:1034] running on GCE node
I20260811 19:03:48.821591 31859 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:03:48.822474 31859 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:03:48.823695 31859 hybrid_clock.cc:648] HybridClock initialized: now 1786475028823674 us; error 30 us; skew 500 ppm
Aug 11 19:03:48 dist-test-slave-0kls krb5kdc[30569](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475028, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:03:48.827447 31859 init.cc:377] Logged in from keytab as kudu/127.13.109.1@KRBTEST.COM (short username kudu)
I20260811 19:03:48.828756 31859 webserver.cc:460] Webserver started at http://127.13.109.1:33679/ using document root <none> and password file <none>
I20260811 19:03:48.829545 31859 fs_manager.cc:359] Metadata directory not provided
I20260811 19:03:48.829648 31859 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:03:48.835090 31859 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 19:03:48.839113 31914 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:03:48.841004 31859 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 19:03:48.841183 31859 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-0/wal
uuid: "0b24c35b827f4b0badf0b66aec15cae9"
format_stamp: "Formatted at 2026-08-11 19:03:06 on dist-test-slave-0kls"
server_key: "1cd0cfa39f884ff13a23f2075b5f2bfa"
server_key_iv: "d92e0b1842bb9ef1b0a5661f58bebd7a"
server_key_version: "encryptionkey@0"
I20260811 19:03:48.842068 31859 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:03:48.869519 31859 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:03:48.874043 31859 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:03:48.874296 31859 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:03:48.875155 31859 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:03:48.876678 31922 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 19:03:48.881316 31859 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 19:03:48.881419 31859 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260811 19:03:48.881500 31859 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 19:03:48.884033 31859 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 19:03:48.884128 31859 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.000s sys 0.001s
I20260811 19:03:48.884562 31922 tablet_bootstrap.cc:492] T e3a9d0813da9442b87967955d0f6b2e9 P 0b24c35b827f4b0badf0b66aec15cae9: Bootstrap starting.
I20260811 19:03:48.896386 31859 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.1:36839
I20260811 19:03:48.896766 32041 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.1:36839 every 8 connection(s)
I20260811 19:03:48.898097 31859 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
I20260811 19:03:48.903322 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 31859
I20260811 19:03:48.903870 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.2:45197
--local_ip_for_outbound_sockets=127.13.109.2
--tserver_master_addrs=127.13.109.62:36875
--webserver_port=36677
--webserver_interface=127.13.109.2
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.2.keytab
--principal=kudu/127.13.109.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:32993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 19:03:48.907997 31922 log.cc:824] T e3a9d0813da9442b87967955d0f6b2e9 P 0b24c35b827f4b0badf0b66aec15cae9: Log is configured to *not* fsync() on all Append() calls
I20260811 19:03:48.920214 31922 tablet_bootstrap.cc:492] T e3a9d0813da9442b87967955d0f6b2e9 P 0b24c35b827f4b0badf0b66aec15cae9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 19:03:48.920737 31922 tablet_bootstrap.cc:492] T e3a9d0813da9442b87967955d0f6b2e9 P 0b24c35b827f4b0badf0b66aec15cae9: Bootstrap complete.
I20260811 19:03:48.922374 31922 ts_tablet_manager.cc:1397] T e3a9d0813da9442b87967955d0f6b2e9 P 0b24c35b827f4b0badf0b66aec15cae9: Time spent bootstrapping tablet: real 0.038s user 0.024s sys 0.011s
I20260811 19:03:48.925921 31922 raft_consensus.cc:348] T e3a9d0813da9442b87967955d0f6b2e9 P 0b24c35b827f4b0badf0b66aec15cae9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2336f52031842798c0dabb12b7e06e6" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 40165 } } peers { permanent_uuid: "e1052c249dde44e6901a8b7840c6e1bb" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 45197 } } peers { permanent_uuid: "0b24c35b827f4b0badf0b66aec15cae9" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 36839 } }
I20260811 19:03:48.926537 31922 raft_consensus.cc:729] T e3a9d0813da9442b87967955d0f6b2e9 P 0b24c35b827f4b0badf0b66aec15cae9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b24c35b827f4b0badf0b66aec15cae9, State: Initialized, Role: FOLLOWER
I20260811 19:03:48.927174 31922 consensus_queue.cc:260] T e3a9d0813da9442b87967955d0f6b2e9 P 0b24c35b827f4b0badf0b66aec15cae9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2336f52031842798c0dabb12b7e06e6" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 40165 } } peers { permanent_uuid: "e1052c249dde44e6901a8b7840c6e1bb" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 45197 } } peers { permanent_uuid: "0b24c35b827f4b0badf0b66aec15cae9" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 36839 } }
I20260811 19:03:48.928483 31922 ts_tablet_manager.cc:1428] T e3a9d0813da9442b87967955d0f6b2e9 P 0b24c35b827f4b0badf0b66aec15cae9: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
W20260811 19:03:48.962020 32044 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:03:48.900799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:36875 (local address 127.13.109.1:50999)
0811 19:03:48.901490 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:48.901559 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0811 19:03:48.902970 (+ 1411us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:03:48.961010 (+ 58040us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:36875: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":189,"threads_started":1}
W20260811 19:03:48.962679 32042 heartbeater.cc:644] Failed to heartbeat to 127.13.109.62:36875 (0 consecutive failures): Network error: Failed to ping master at 127.13.109.62:36875: Client connection negotiation failed: client connection to 127.13.109.62:36875: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 19:03:48.963522 32044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:48.963226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:36875 (local address 127.13.109.1:43285)
0811 19:03:48.963367 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:48.963452 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:36875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260811 19:03:48.964226 32044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:48.963991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:36875 (local address 127.13.109.1:53619)
0811 19:03:48.964119 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:48.964171 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:36875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260811 19:03:48.964411 32042 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 19:03:49.044368 32045 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:03:49.044715 32045 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:03:49.044796 32045 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:03:49.049289 32045 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:03:49.049396 32045 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:03:49.049497 32045 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.2
I20260811 19:03:49.055397 32045 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:32993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.2.keytab
--principal=kudu/127.13.109.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.2:45197
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.109.2
--webserver_port=36677
--tserver_master_addrs=127.13.109.62:36875
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32045
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.2
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:03:49.056684 32045 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:03:49.057828 32045 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:03:49.068789 32054 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:03:49.068951 32045 server_base.cc:1034] running on GCE node
W20260811 19:03:49.068791 32053 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:03:49.069190 32056 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:03:49.069674 32045 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:03:49.070489 32045 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:03:49.071666 32045 hybrid_clock.cc:648] HybridClock initialized: now 1786475029071652 us; error 30 us; skew 500 ppm
Aug 11 19:03:49 dist-test-slave-0kls krb5kdc[30569](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475029, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:03:49.074826 32045 init.cc:377] Logged in from keytab as kudu/127.13.109.2@KRBTEST.COM (short username kudu)
I20260811 19:03:49.075870 32045 webserver.cc:460] Webserver started at http://127.13.109.2:36677/ using document root <none> and password file <none>
I20260811 19:03:49.076498 32045 fs_manager.cc:359] Metadata directory not provided
I20260811 19:03:49.076573 32045 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:03:49.081059 32045 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 19:03:49.083455 32063 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:03:49.084750 32045 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 19:03:49.084903 32045 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-1/wal
uuid: "e1052c249dde44e6901a8b7840c6e1bb"
format_stamp: "Formatted at 2026-08-11 19:03:06 on dist-test-slave-0kls"
server_key: "6383c25d81e363f2d5dab237d58d0b1d"
server_key_iv: "9d9dc05dcc7917ea22074ebcd087da87"
server_key_version: "encryptionkey@0"
I20260811 19:03:49.085492 32045 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:03:49.112980 32045 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:03:49.116531 32045 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:03:49.116768 32045 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:03:49.117481 32045 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:03:49.118474 32045 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:03:49.118531 32045 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:03:49.118584 32045 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:03:49.118645 32045 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:03:49.128515 32045 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.2:45197
I20260811 19:03:49.128535 32192 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.2:45197 every 8 connection(s)
I20260811 19:03:49.129606 32045 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
I20260811 19:03:49.133545 32195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:49.131927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:36875 (local address 127.13.109.2:37071)
0811 19:03:49.132658 (+ 731us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:49.132775 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:36875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":531,"thread_start_us":133,"threads_started":1}
W20260811 19:03:49.134097 32193 heartbeater.cc:644] Failed to heartbeat to 127.13.109.62:36875 (0 consecutive failures): Network error: Failed to ping master at 127.13.109.62:36875: Client connection negotiation failed: client connection to 127.13.109.62:36875: connect: Connection refused (error 111)
I20260811 19:03:49.134881 32195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:49.134603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:36875 (local address 127.13.109.2:46271)
0811 19:03:49.134752 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:49.134809 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:36875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260811 19:03:49.135036 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 32045
I20260811 19:03:49.135659 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.3:40165
--local_ip_for_outbound_sockets=127.13.109.3
--tserver_master_addrs=127.13.109.62:36875
--webserver_port=37157
--webserver_interface=127.13.109.3
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.3.keytab
--principal=kudu/127.13.109.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:32993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 19:03:49.135771 32195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:49.135491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:36875 (local address 127.13.109.2:50429)
0811 19:03:49.135633 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:49.135680 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:36875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260811 19:03:49.136112 32193 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 19:03:49.257339 32196 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:03:49.257685 32196 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:03:49.257735 32196 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:03:49.262133 32196 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:03:49.262233 32196 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:03:49.262297 32196 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.3
I20260811 19:03:49.268293 32196 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:32993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.3.keytab
--principal=kudu/127.13.109.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.3:40165
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.109.3
--webserver_port=37157
--tserver_master_addrs=127.13.109.62:36875
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32196
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.3
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:03:49.269487 32196 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:03:49.270557 32196 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:03:49.281057 32204 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:03:49.281057 32202 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:03:49.281147 32207 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:03:49.281395 32196 server_base.cc:1034] running on GCE node
I20260811 19:03:49.282013 32196 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:03:49.282771 32196 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:03:49.283958 32196 hybrid_clock.cc:648] HybridClock initialized: now 1786475029283915 us; error 55 us; skew 500 ppm
Aug 11 19:03:49 dist-test-slave-0kls krb5kdc[30569](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475029, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:03:49.287027 32196 init.cc:377] Logged in from keytab as kudu/127.13.109.3@KRBTEST.COM (short username kudu)
I20260811 19:03:49.288038 32196 webserver.cc:460] Webserver started at http://127.13.109.3:37157/ using document root <none> and password file <none>
I20260811 19:03:49.288714 32196 fs_manager.cc:359] Metadata directory not provided
I20260811 19:03:49.288821 32196 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:03:49.293071 32196 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 19:03:49.295456 32214 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:03:49.296810 32196 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 19:03:49.296963 32196 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-2/wal
uuid: "a2336f52031842798c0dabb12b7e06e6"
format_stamp: "Formatted at 2026-08-11 19:03:06 on dist-test-slave-0kls"
server_key: "7a614070ce0fa326c55a07c0c28a6cf1"
server_key_iv: "f0757db09afeab1fd0cf5f0af21d1f3e"
server_key_version: "encryptionkey@0"
I20260811 19:03:49.297567 32196 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:03:49.318663 32196 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:03:49.322523 32196 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:03:49.322780 32196 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:03:49.323521 32196 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:03:49.324478 32196 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:03:49.324563 32196 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:03:49.324659 32196 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:03:49.324714 32196 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:03:49.334524 32196 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.3:40165
I20260811 19:03:49.334616 32336 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.3:40165 every 8 connection(s)
I20260811 19:03:49.335687 32196 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
I20260811 19:03:49.339790 32340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:49.338158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:36875 (local address 127.13.109.3:41721)
0811 19:03:49.338907 (+ 749us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:49.339044 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:36875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":123,"threads_started":1}
W20260811 19:03:49.340288 32337 heartbeater.cc:644] Failed to heartbeat to 127.13.109.62:36875 (0 consecutive failures): Network error: Failed to ping master at 127.13.109.62:36875: Client connection negotiation failed: client connection to 127.13.109.62:36875: connect: Connection refused (error 111)
I20260811 19:03:49.341013 32340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:49.340778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:36875 (local address 127.13.109.3:34285)
0811 19:03:49.340894 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:49.340947 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:36875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260811 19:03:49.341590 32340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:49.341377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:36875 (local address 127.13.109.3:51663)
0811 19:03:49.341486 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:49.341530 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:36875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260811 19:03:49.341768 32337 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 19:03:49.344663 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 32196
I20260811 19:03:49.965701 32343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:49.965111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:36875 (local address 127.13.109.1:48511)
0811 19:03:49.965471 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:49.965552 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:36875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":129,"threads_started":1}
I20260811 19:03:50.137313 32344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:50.136845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:36875 (local address 127.13.109.2:40845)
0811 19:03:50.137131 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:50.137198 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:36875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":79,"threads_started":1}
I20260811 19:03:50.342981 32345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:50.342449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:36875 (local address 127.13.109.3:58641)
0811 19:03:50.342788 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:50.342867 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:36875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":79,"threads_started":1}
I20260811 19:03:50.721702 32348 raft_consensus.cc:482] T e3a9d0813da9442b87967955d0f6b2e9 P 0b24c35b827f4b0badf0b66aec15cae9 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:03:50.721982 32348 raft_consensus.cc:504] T e3a9d0813da9442b87967955d0f6b2e9 P 0b24c35b827f4b0badf0b66aec15cae9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2336f52031842798c0dabb12b7e06e6" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 40165 } } peers { permanent_uuid: "e1052c249dde44e6901a8b7840c6e1bb" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 45197 } } peers { permanent_uuid: "0b24c35b827f4b0badf0b66aec15cae9" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 36839 } }
I20260811 19:03:50.723135 32348 leader_election.cc:288] T e3a9d0813da9442b87967955d0f6b2e9 P 0b24c35b827f4b0badf0b66aec15cae9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2336f52031842798c0dabb12b7e06e6 (127.13.109.3:40165), e1052c249dde44e6901a8b7840c6e1bb (127.13.109.2:45197)
Aug 11 19:03:50 dist-test-slave-0kls krb5kdc[30569](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475028, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.1@KRBTEST.COM for kudu/127.13.109.2@KRBTEST.COM
Aug 11 19:03:50 dist-test-slave-0kls krb5kdc[30569](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475028, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.1@KRBTEST.COM for kudu/127.13.109.3@KRBTEST.COM
I20260811 19:03:50.738399 32352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:50.723619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.1:56685 (local address 127.13.109.3:40165)
0811 19:03:50.724203 (+ 584us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:50.724224 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:50.724259 (+ 35us) server_negotiation.cc:407] Connection header received
0811 19:03:50.724912 (+ 653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:50.724918 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:50.725228 (+ 310us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:50.725532 (+ 304us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:50.726987 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:50.728244 (+ 1257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:50.729629 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:50.730218 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:50.730423 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:50.733723 (+ 3300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:03:50.733727 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:03:50.733738 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:03:50.733769 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:03:50.735855 (+ 2086us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:50.736412 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:50.736417 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:50.736423 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:50.736551 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:50.736862 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:50.736864 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:50.736865 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:50.737276 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:03:50.737410 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:50.737910 (+ 500us) server_negotiation.cc:299] Negotiation successful
0811 19:03:50.738192 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":469,"thread_start_us":81,"threads_started":1}
I20260811 19:03:50.738421 32353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:50.723624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.1:41345 (local address 127.13.109.2:45197)
0811 19:03:50.724074 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:50.724088 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:50.724129 (+ 41us) server_negotiation.cc:407] Connection header received
0811 19:03:50.724816 (+ 687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:50.724822 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:50.725200 (+ 378us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:50.725473 (+ 273us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:50.726988 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:50.728249 (+ 1261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:50.729630 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:50.730216 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:50.730422 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:50.733420 (+ 2998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:03:50.733430 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:03:50.733451 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:03:50.733493 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:03:50.735385 (+ 1892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:50.736040 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:50.736046 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:50.736052 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:50.736217 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:50.736660 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:50.736663 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:50.736665 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:50.737187 (+ 522us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:03:50.737296 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:50.737910 (+ 614us) server_negotiation.cc:299] Negotiation successful
0811 19:03:50.738229 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":115,"threads_started":1}
I20260811 19:03:50.738426 32350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:50.723428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.2:45197 (local address 127.13.109.1:41345)
0811 19:03:50.723790 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:50.723808 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 19:03:50.723936 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:03:50.725921 (+ 1985us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:03:50.725926 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:03:50.726203 (+ 277us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:03:50.726775 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:03:50.726792 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:03:50.728430 (+ 1638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:03:50.728433 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:03:50.729469 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:03:50.729481 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:03:50.730359 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:03:50.730363 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:03:50.730650 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:50.731564 (+ 914us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:03:50.731576 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:03:50.733148 (+ 1572us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:03:50.735524 (+ 2376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:03:50.735531 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:03:50.735547 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:03:50.735893 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:03:50.736332 (+ 439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:03:50.736334 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:03:50.736335 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:03:50.736543 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:03:50.737317 (+ 774us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:03:50.737327 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:03:50.737573 (+ 246us) client_negotiation.cc:769] Sending connection context
0811 19:03:50.737765 (+ 192us) client_negotiation.cc:240] Negotiation successful
0811 19:03:50.738192 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"mutex_wait_us":166,"spinlock_wait_cycles":42880,"thread_start_us":84,"threads_started":1}
I20260811 19:03:50.738770 32349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:50.723428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.3:40165 (local address 127.13.109.1:56685)
0811 19:03:50.723885 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:50.723898 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 19:03:50.723963 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:03:50.725619 (+ 1656us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:03:50.725626 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:03:50.726208 (+ 582us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:03:50.726775 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:03:50.726793 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:03:50.728419 (+ 1626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:03:50.728423 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:03:50.729471 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:03:50.729481 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:03:50.730359 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:03:50.730363 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:03:50.730652 (+ 289us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:50.731490 (+ 838us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:03:50.731521 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:03:50.733510 (+ 1989us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:03:50.736022 (+ 2512us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:03:50.736025 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:03:50.736027 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:03:50.736296 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:03:50.736670 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:03:50.736674 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:03:50.736675 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:03:50.736761 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:03:50.737397 (+ 636us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:03:50.737399 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:03:50.737567 (+ 168us) client_negotiation.cc:769] Sending connection context
0811 19:03:50.737767 (+ 200us) client_negotiation.cc:240] Negotiation successful
0811 19:03:50.738282 (+ 515us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":390,"mutex_wait_us":403,"spinlock_wait_cycles":29312,"thread_start_us":66,"threads_started":1}
W20260811 19:03:50.741201 31916 leader_election.cc:341] T e3a9d0813da9442b87967955d0f6b2e9 P 0b24c35b827f4b0badf0b66aec15cae9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e1052c249dde44e6901a8b7840c6e1bb (127.13.109.2:45197): Not found: Tablet not found: e3a9d0813da9442b87967955d0f6b2e9
W20260811 19:03:50.741374 31915 leader_election.cc:341] T e3a9d0813da9442b87967955d0f6b2e9 P 0b24c35b827f4b0badf0b66aec15cae9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a2336f52031842798c0dabb12b7e06e6 (127.13.109.3:40165): Not found: Tablet not found: e3a9d0813da9442b87967955d0f6b2e9
I20260811 19:03:50.741464 31915 leader_election.cc:302] T e3a9d0813da9442b87967955d0f6b2e9 P 0b24c35b827f4b0badf0b66aec15cae9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0b24c35b827f4b0badf0b66aec15cae9; no voters: a2336f52031842798c0dabb12b7e06e6, e1052c249dde44e6901a8b7840c6e1bb
I20260811 19:03:50.741719 32348 raft_consensus.cc:2726] T e3a9d0813da9442b87967955d0f6b2e9 P 0b24c35b827f4b0badf0b66aec15cae9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 19:03:50.967079 32350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:50.966706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:36875 (local address 127.13.109.1:49635)
0811 19:03:50.966897 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:50.966967 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:36875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260811 19:03:51.138762 32354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:51.138257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:36875 (local address 127.13.109.2:56761)
0811 19:03:51.138547 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:51.138614 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:36875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":84,"threads_started":1}
I20260811 19:03:51.344328 32355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:51.343836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:36875 (local address 127.13.109.3:34295)
0811 19:03:51.344141 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:51.344206 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:36875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":82,"threads_started":1}
I20260811 19:03:51.968396 32356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:51.967974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:36875 (local address 127.13.109.1:49637)
0811 19:03:51.968239 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:51.968295 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:36875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":102,"threads_started":1}
I20260811 19:03:52.140437 32357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:52.139873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:36875 (local address 127.13.109.2:34063)
0811 19:03:52.140258 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:52.140332 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:36875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":92,"threads_started":1}
I20260811 19:03:52.345880 32359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:52.345385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:36875 (local address 127.13.109.3:42401)
0811 19:03:52.345661 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:52.345725 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:36875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":87,"threads_started":1}
I20260811 19:03:52.739198 32360 raft_consensus.cc:482] T e3a9d0813da9442b87967955d0f6b2e9 P 0b24c35b827f4b0badf0b66aec15cae9 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:03:52.739393 32360 raft_consensus.cc:504] T e3a9d0813da9442b87967955d0f6b2e9 P 0b24c35b827f4b0badf0b66aec15cae9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2336f52031842798c0dabb12b7e06e6" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 40165 } } peers { permanent_uuid: "e1052c249dde44e6901a8b7840c6e1bb" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 45197 } } peers { permanent_uuid: "0b24c35b827f4b0badf0b66aec15cae9" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 36839 } }
I20260811 19:03:52.739926 32360 leader_election.cc:288] T e3a9d0813da9442b87967955d0f6b2e9 P 0b24c35b827f4b0badf0b66aec15cae9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2336f52031842798c0dabb12b7e06e6 (127.13.109.3:40165), e1052c249dde44e6901a8b7840c6e1bb (127.13.109.2:45197)
W20260811 19:03:52.740844 31916 leader_election.cc:341] T e3a9d0813da9442b87967955d0f6b2e9 P 0b24c35b827f4b0badf0b66aec15cae9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e1052c249dde44e6901a8b7840c6e1bb (127.13.109.2:45197): Not found: Tablet not found: e3a9d0813da9442b87967955d0f6b2e9
W20260811 19:03:52.740942 31915 leader_election.cc:341] T e3a9d0813da9442b87967955d0f6b2e9 P 0b24c35b827f4b0badf0b66aec15cae9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a2336f52031842798c0dabb12b7e06e6 (127.13.109.3:40165): Not found: Tablet not found: e3a9d0813da9442b87967955d0f6b2e9
I20260811 19:03:52.741058 31915 leader_election.cc:302] T e3a9d0813da9442b87967955d0f6b2e9 P 0b24c35b827f4b0badf0b66aec15cae9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0b24c35b827f4b0badf0b66aec15cae9; no voters: a2336f52031842798c0dabb12b7e06e6, e1052c249dde44e6901a8b7840c6e1bb
I20260811 19:03:52.741276 32360 raft_consensus.cc:2726] T e3a9d0813da9442b87967955d0f6b2e9 P 0b24c35b827f4b0badf0b66aec15cae9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 19:03:52.969653 32361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:52.969142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:36875 (local address 127.13.109.1:53913)
0811 19:03:52.969456 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:52.969523 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:36875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":83,"threads_started":1}
I20260811 19:03:53.141899 32362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:53.141352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:36875 (local address 127.13.109.2:36895)
0811 19:03:53.141664 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:53.141731 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:36875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":97,"threads_started":1}
I20260811 19:03:53.347293 32363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:53.346780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:36875 (local address 127.13.109.3:48003)
0811 19:03:53.347110 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:53.347187 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:36875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":123,"threads_started":1}
I20260811 19:03:53.971092 32364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:53.970621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:36875 (local address 127.13.109.1:51309)
0811 19:03:53.970932 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:53.970992 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:36875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":83,"threads_started":1}
I20260811 19:03:54.143332 32366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:54.142824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:36875 (local address 127.13.109.2:34649)
0811 19:03:54.143154 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:54.143223 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:36875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":82,"threads_started":1}
I20260811 19:03:54.348767 32367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:54.348247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:36875 (local address 127.13.109.3:38863)
0811 19:03:54.348606 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:54.348668 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:36875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":114,"threads_started":1}
I20260811 19:03:54.972374 32368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:54.971930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:36875 (local address 127.13.109.1:35931)
0811 19:03:54.972214 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:54.972273 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:36875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":82,"threads_started":1}
I20260811 19:03:55.144819 32369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:55.144305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:36875 (local address 127.13.109.2:37123)
0811 19:03:55.144645 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:55.144713 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:36875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":102,"threads_started":1}
I20260811 19:03:55.349985 32370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:55.349506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:36875 (local address 127.13.109.3:40221)
0811 19:03:55.349819 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:55.349884 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:36875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":92,"threads_started":1}
I20260811 19:03:55.695101 32371 raft_consensus.cc:482] T e3a9d0813da9442b87967955d0f6b2e9 P 0b24c35b827f4b0badf0b66aec15cae9 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:03:55.695240 32371 raft_consensus.cc:504] T e3a9d0813da9442b87967955d0f6b2e9 P 0b24c35b827f4b0badf0b66aec15cae9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2336f52031842798c0dabb12b7e06e6" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 40165 } } peers { permanent_uuid: "e1052c249dde44e6901a8b7840c6e1bb" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 45197 } } peers { permanent_uuid: "0b24c35b827f4b0badf0b66aec15cae9" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 36839 } }
I20260811 19:03:55.695729 32371 leader_election.cc:288] T e3a9d0813da9442b87967955d0f6b2e9 P 0b24c35b827f4b0badf0b66aec15cae9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2336f52031842798c0dabb12b7e06e6 (127.13.109.3:40165), e1052c249dde44e6901a8b7840c6e1bb (127.13.109.2:45197)
W20260811 19:03:55.696578 31915 leader_election.cc:341] T e3a9d0813da9442b87967955d0f6b2e9 P 0b24c35b827f4b0badf0b66aec15cae9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a2336f52031842798c0dabb12b7e06e6 (127.13.109.3:40165): Not found: Tablet not found: e3a9d0813da9442b87967955d0f6b2e9
W20260811 19:03:55.696671 31916 leader_election.cc:341] T e3a9d0813da9442b87967955d0f6b2e9 P 0b24c35b827f4b0badf0b66aec15cae9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e1052c249dde44e6901a8b7840c6e1bb (127.13.109.2:45197): Not found: Tablet not found: e3a9d0813da9442b87967955d0f6b2e9
I20260811 19:03:55.696784 31916 leader_election.cc:302] T e3a9d0813da9442b87967955d0f6b2e9 P 0b24c35b827f4b0badf0b66aec15cae9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0b24c35b827f4b0badf0b66aec15cae9; no voters: a2336f52031842798c0dabb12b7e06e6, e1052c249dde44e6901a8b7840c6e1bb
I20260811 19:03:55.696975 32371 raft_consensus.cc:2726] T e3a9d0813da9442b87967955d0f6b2e9 P 0b24c35b827f4b0badf0b66aec15cae9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 19:03:55.973850 32372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:55.973315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:36875 (local address 127.13.109.1:54571)
0811 19:03:55.973627 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:55.973689 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:36875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":99,"threads_started":1}
I20260811 19:03:56.146409 32373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:56.145883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:36875 (local address 127.13.109.2:33599)
0811 19:03:56.146237 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:56.146305 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:36875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":77,"threads_started":1}
I20260811 19:03:56.351372 32374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:56.350830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:36875 (local address 127.13.109.3:55665)
0811 19:03:56.351184 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:56.351269 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:36875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":90,"threads_started":1}
I20260811 19:03:56.975323 32375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:56.974833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:36875 (local address 127.13.109.1:39423)
0811 19:03:56.975134 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:56.975219 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:36875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":86,"threads_started":1}
I20260811 19:03:57.147666 32376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:57.147225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:36875 (local address 127.13.109.2:56499)
0811 19:03:57.147511 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:57.147568 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:36875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":102,"threads_started":1}
I20260811 19:03:57.352744 32377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:57.352270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:36875 (local address 127.13.109.3:59857)
0811 19:03:57.352575 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:57.352641 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:36875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":99,"threads_started":1}
I20260811 19:03:57.358084 32378 raft_consensus.cc:482] T e3a9d0813da9442b87967955d0f6b2e9 P 0b24c35b827f4b0badf0b66aec15cae9 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:03:57.358246 32378 raft_consensus.cc:504] T e3a9d0813da9442b87967955d0f6b2e9 P 0b24c35b827f4b0badf0b66aec15cae9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2336f52031842798c0dabb12b7e06e6" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 40165 } } peers { permanent_uuid: "e1052c249dde44e6901a8b7840c6e1bb" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 45197 } } peers { permanent_uuid: "0b24c35b827f4b0badf0b66aec15cae9" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 36839 } }
I20260811 19:03:57.358703 32378 leader_election.cc:288] T e3a9d0813da9442b87967955d0f6b2e9 P 0b24c35b827f4b0badf0b66aec15cae9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2336f52031842798c0dabb12b7e06e6 (127.13.109.3:40165), e1052c249dde44e6901a8b7840c6e1bb (127.13.109.2:45197)
W20260811 19:03:57.359510 31915 leader_election.cc:341] T e3a9d0813da9442b87967955d0f6b2e9 P 0b24c35b827f4b0badf0b66aec15cae9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a2336f52031842798c0dabb12b7e06e6 (127.13.109.3:40165): Not found: Tablet not found: e3a9d0813da9442b87967955d0f6b2e9
W20260811 19:03:57.359611 31916 leader_election.cc:341] T e3a9d0813da9442b87967955d0f6b2e9 P 0b24c35b827f4b0badf0b66aec15cae9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e1052c249dde44e6901a8b7840c6e1bb (127.13.109.2:45197): Not found: Tablet not found: e3a9d0813da9442b87967955d0f6b2e9
I20260811 19:03:57.359719 31916 leader_election.cc:302] T e3a9d0813da9442b87967955d0f6b2e9 P 0b24c35b827f4b0badf0b66aec15cae9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0b24c35b827f4b0badf0b66aec15cae9; no voters: a2336f52031842798c0dabb12b7e06e6, e1052c249dde44e6901a8b7840c6e1bb
I20260811 19:03:57.359913 32378 raft_consensus.cc:2726] T e3a9d0813da9442b87967955d0f6b2e9 P 0b24c35b827f4b0badf0b66aec15cae9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 19:03:57.976816 32379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:57.976304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:36875 (local address 127.13.109.1:49635)
0811 19:03:57.976632 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:57.976710 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:36875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":105,"threads_started":1}
I20260811 19:03:58.148985 32380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:58.148465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:36875 (local address 127.13.109.2:37831)
0811 19:03:58.148813 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:58.148874 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:36875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":96,"threads_started":1}
I20260811 19:03:58.354229 32381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:58.353680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:36875 (local address 127.13.109.3:42345)
0811 19:03:58.354016 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:58.354113 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:36875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":140,"threads_started":1}
I20260811 19:03:58.944612 32382 raft_consensus.cc:482] T e3a9d0813da9442b87967955d0f6b2e9 P 0b24c35b827f4b0badf0b66aec15cae9 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:03:58.944774 32382 raft_consensus.cc:504] T e3a9d0813da9442b87967955d0f6b2e9 P 0b24c35b827f4b0badf0b66aec15cae9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2336f52031842798c0dabb12b7e06e6" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 40165 } } peers { permanent_uuid: "e1052c249dde44e6901a8b7840c6e1bb" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 45197 } } peers { permanent_uuid: "0b24c35b827f4b0badf0b66aec15cae9" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 36839 } }
I20260811 19:03:58.945194 32382 leader_election.cc:288] T e3a9d0813da9442b87967955d0f6b2e9 P 0b24c35b827f4b0badf0b66aec15cae9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2336f52031842798c0dabb12b7e06e6 (127.13.109.3:40165), e1052c249dde44e6901a8b7840c6e1bb (127.13.109.2:45197)
W20260811 19:03:58.946084 31916 leader_election.cc:341] T e3a9d0813da9442b87967955d0f6b2e9 P 0b24c35b827f4b0badf0b66aec15cae9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e1052c249dde44e6901a8b7840c6e1bb (127.13.109.2:45197): Not found: Tablet not found: e3a9d0813da9442b87967955d0f6b2e9
W20260811 19:03:58.946188 31915 leader_election.cc:341] T e3a9d0813da9442b87967955d0f6b2e9 P 0b24c35b827f4b0badf0b66aec15cae9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a2336f52031842798c0dabb12b7e06e6 (127.13.109.3:40165): Not found: Tablet not found: e3a9d0813da9442b87967955d0f6b2e9
I20260811 19:03:58.946254 31915 leader_election.cc:302] T e3a9d0813da9442b87967955d0f6b2e9 P 0b24c35b827f4b0badf0b66aec15cae9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0b24c35b827f4b0badf0b66aec15cae9; no voters: a2336f52031842798c0dabb12b7e06e6, e1052c249dde44e6901a8b7840c6e1bb
I20260811 19:03:58.946398 32382 raft_consensus.cc:2726] T e3a9d0813da9442b87967955d0f6b2e9 P 0b24c35b827f4b0badf0b66aec15cae9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 19:03:58.978302 32383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:58.977857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:36875 (local address 127.13.109.1:45641)
0811 19:03:58.978134 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:58.978186 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:36875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":96,"threads_started":1}
I20260811 19:03:59.150414 32384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:59.149978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:36875 (local address 127.13.109.2:60023)
0811 19:03:59.150246 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:59.150310 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:36875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":76,"threads_started":1}
I20260811 19:03:59.355839 32385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:59.355366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:36875 (local address 127.13.109.3:40855)
0811 19:03:59.355674 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:59.355735 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:36875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":116,"threads_started":1}
I20260811 19:03:59.979993 32386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:59.979454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:36875 (local address 127.13.109.1:46243)
0811 19:03:59.979823 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:59.979890 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:36875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":116,"threads_started":1}
I20260811 19:04:00.151681 32387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:00.151230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:36875 (local address 127.13.109.2:40517)
0811 19:04:00.151500 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:00.151578 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:36875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":73,"threads_started":1}
I20260811 19:04:00.357272 32388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:00.356764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:36875 (local address 127.13.109.3:52337)
0811 19:04:00.357104 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:00.357171 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:36875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":104,"threads_started":1}
I20260811 19:04:00.945340 32389 raft_consensus.cc:482] T e3a9d0813da9442b87967955d0f6b2e9 P 0b24c35b827f4b0badf0b66aec15cae9 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:04:00.945546 32389 raft_consensus.cc:504] T e3a9d0813da9442b87967955d0f6b2e9 P 0b24c35b827f4b0badf0b66aec15cae9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2336f52031842798c0dabb12b7e06e6" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 40165 } } peers { permanent_uuid: "e1052c249dde44e6901a8b7840c6e1bb" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 45197 } } peers { permanent_uuid: "0b24c35b827f4b0badf0b66aec15cae9" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 36839 } }
I20260811 19:04:00.946043 32389 leader_election.cc:288] T e3a9d0813da9442b87967955d0f6b2e9 P 0b24c35b827f4b0badf0b66aec15cae9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2336f52031842798c0dabb12b7e06e6 (127.13.109.3:40165), e1052c249dde44e6901a8b7840c6e1bb (127.13.109.2:45197)
W20260811 19:04:00.946892 31915 leader_election.cc:341] T e3a9d0813da9442b87967955d0f6b2e9 P 0b24c35b827f4b0badf0b66aec15cae9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a2336f52031842798c0dabb12b7e06e6 (127.13.109.3:40165): Not found: Tablet not found: e3a9d0813da9442b87967955d0f6b2e9
W20260811 19:04:00.946981 31916 leader_election.cc:341] T e3a9d0813da9442b87967955d0f6b2e9 P 0b24c35b827f4b0badf0b66aec15cae9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e1052c249dde44e6901a8b7840c6e1bb (127.13.109.2:45197): Not found: Tablet not found: e3a9d0813da9442b87967955d0f6b2e9
I20260811 19:04:00.947113 31916 leader_election.cc:302] T e3a9d0813da9442b87967955d0f6b2e9 P 0b24c35b827f4b0badf0b66aec15cae9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0b24c35b827f4b0badf0b66aec15cae9; no voters: a2336f52031842798c0dabb12b7e06e6, e1052c249dde44e6901a8b7840c6e1bb
I20260811 19:04:00.947309 32389 raft_consensus.cc:2726] T e3a9d0813da9442b87967955d0f6b2e9 P 0b24c35b827f4b0badf0b66aec15cae9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 19:04:00.981433 32390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:00.980959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:36875 (local address 127.13.109.1:51661)
0811 19:04:00.981266 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:00.981332 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:36875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":104,"threads_started":1}
I20260811 19:04:01.153098 32391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:01.152592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:36875 (local address 127.13.109.2:42565)
0811 19:04:01.152929 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:01.152993 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:36875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":105,"threads_started":1}
I20260811 19:04:01.358693 32392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:01.358232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:36875 (local address 127.13.109.3:55957)
0811 19:04:01.358529 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:01.358586 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:36875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":95,"threads_started":1}
I20260811 19:04:01.983240 32393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:01.982586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:36875 (local address 127.13.109.1:42037)
0811 19:04:01.983016 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:01.983093 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:36875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":101,"threads_started":1}
I20260811 19:04:02.154533 32394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:02.154078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:36875 (local address 127.13.109.2:41043)
0811 19:04:02.154344 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:02.154414 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:36875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":75,"threads_started":1}
I20260811 19:04:02.360342 32395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:02.359788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:36875 (local address 127.13.109.3:57189)
0811 19:04:02.360165 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:02.360235 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:36875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":110,"threads_started":1}
I20260811 19:04:02.984792 32396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:02.984218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:36875 (local address 127.13.109.1:43139)
0811 19:04:02.984578 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:02.984668 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:36875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":89,"threads_started":1}
I20260811 19:04:03.155822 32397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:03.155353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:36875 (local address 127.13.109.2:45757)
0811 19:04:03.155654 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:03.155717 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:36875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":109,"threads_started":1}
I20260811 19:04:03.361941 32398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:03.361395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:36875 (local address 127.13.109.3:54211)
0811 19:04:03.361702 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:03.361763 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:36875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":110,"threads_started":1}
I20260811 19:04:03.986639 32399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:03.986126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:36875 (local address 127.13.109.1:56035)
0811 19:04:03.986449 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:03.986526 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:36875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":78,"threads_started":1}
I20260811 19:04:04.157337 32400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:04.156860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:36875 (local address 127.13.109.2:54717)
0811 19:04:04.157143 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:04.157227 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:36875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":98,"threads_started":1}
I20260811 19:04:04.348910 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 31859
I20260811 19:04:04.358259 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 32045
I20260811 19:04:04.363672 32401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:04.363122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:36875 (local address 127.13.109.3:55805)
0811 19:04:04.363471 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:04.363555 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.62:36875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":133,"threads_started":1}
I20260811 19:04:04.368234 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 32196
2026-08-11T19:04:04Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59957 ms)
[----------] 30 tests from SecurityITest (251867 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:04:04 dist-test-slave-0kls krb5kdc[32404](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:04:04 dist-test-slave-0kls krb5kdc[32404](info): set up 2 sockets
Aug 11 19:04:04 dist-test-slave-0kls krb5kdc[32404](info): commencing operation
krb5kdc: starting...
W20260811 19:04:05.478116 13748 mini_kdc.cc:121] Time spent starting KDC: real 1.082s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/krb5kdc/test-user.keytab.
Aug 11 19:04:05 dist-test-slave-0kls krb5kdc[32404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475045, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:04:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:04:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.109.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.109.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.109.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab.
Entry for principal kudu/127.13.109.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab.
I20260811 19:04:05.620776 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.62:35737
--webserver_interface=127.13.109.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab
--principal=kudu/127.13.109.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:37471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.109.62:35737
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:04:05.741043 32419 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:04:05.741422 32419 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:04:05.741508 32419 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:04:05.745704 32419 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:04:05.745862 32419 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:04:05.745905 32419 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:04:05.745939 32419 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:04:05.745970 32419 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:04:05.751789 32419 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:37471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.109.62:35737
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab
--principal=kudu/127.13.109.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.62:35737
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.109.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32419
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:04:05.753064 32419 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:04:05.754200 32419 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:04:05.763209 32425 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:04:05.763216 32428 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:04:05.763288 32426 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:04:05.763468 32419 server_base.cc:1034] running on GCE node
I20260811 19:04:05.764129 32419 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:04:05.765177 32419 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:04:05.766378 32419 hybrid_clock.cc:648] HybridClock initialized: now 1786475045766358 us; error 30 us; skew 500 ppm
Aug 11 19:04:05 dist-test-slave-0kls krb5kdc[32404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475045, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:04:05.769140 32419 init.cc:377] Logged in from keytab as kudu/127.13.109.62@KRBTEST.COM (short username kudu)
I20260811 19:04:05.770146 32419 webserver.cc:460] Webserver started at http://127.13.109.62:35569/ using document root <none> and password file <none>
I20260811 19:04:05.770680 32419 fs_manager.cc:359] Metadata directory not provided
I20260811 19:04:05.770741 32419 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:04:05.770915 32419 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:04:05.773015 32419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/minicluster-data/master-0/data/instance:
uuid: "efd74634030b45369f435df37f3caa07"
format_stamp: "Formatted at 2026-08-11 19:04:05 on dist-test-slave-0kls"
server_key: "0239092e04761bb57f059a4e775d2114"
server_key_iv: "bdceea7b45474c0a3d52eb0a51226658"
server_key_version: "encryptionkey@0"
I20260811 19:04:05.773535 32419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/minicluster-data/master-0/wal/instance:
uuid: "efd74634030b45369f435df37f3caa07"
format_stamp: "Formatted at 2026-08-11 19:04:05 on dist-test-slave-0kls"
server_key: "0239092e04761bb57f059a4e775d2114"
server_key_iv: "bdceea7b45474c0a3d52eb0a51226658"
server_key_version: "encryptionkey@0"
I20260811 19:04:05.777349 32419 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 19:04:05.779891 32436 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:04:05.781155 32419 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 19:04:05.781261 32419 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/minicluster-data/master-0/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/minicluster-data/master-0/wal
uuid: "efd74634030b45369f435df37f3caa07"
format_stamp: "Formatted at 2026-08-11 19:04:05 on dist-test-slave-0kls"
server_key: "0239092e04761bb57f059a4e775d2114"
server_key_iv: "bdceea7b45474c0a3d52eb0a51226658"
server_key_version: "encryptionkey@0"
I20260811 19:04:05.781350 32419 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:04:05.827227 32419 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:04:05.830325 32419 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:04:05.830492 32419 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:04:05.838315 32419 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.62:35737
I20260811 19:04:05.838310 32495 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.62:35737 every 8 connection(s)
I20260811 19:04:05.839388 32419 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
I20260811 19:04:05.842533 32496 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:04:05.848513 32496 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P efd74634030b45369f435df37f3caa07: Bootstrap starting.
I20260811 19:04:05.849601 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 32419
I20260811 19:04:05.849713 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/minicluster-data/master-0/wal/instance
I20260811 19:04:05.850150 13748 external_mini_cluster.cc:1428] Setting key 281323042e5c319f552fb0645d770b3e
I20260811 19:04:05.851539 32496 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P efd74634030b45369f435df37f3caa07: Neither blocks nor log segments found. Creating new log.
I20260811 19:04:05.852403 32496 log.cc:824] T 00000000000000000000000000000000 P efd74634030b45369f435df37f3caa07: Log is configured to *not* fsync() on all Append() calls
I20260811 19:04:05.854557 32496 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P efd74634030b45369f435df37f3caa07: No bootstrap required, opened a new log
I20260811 19:04:05.858390 32496 raft_consensus.cc:348] T 00000000000000000000000000000000 P efd74634030b45369f435df37f3caa07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efd74634030b45369f435df37f3caa07" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 35737 } }
I20260811 19:04:05.858704 32496 raft_consensus.cc:374] T 00000000000000000000000000000000 P efd74634030b45369f435df37f3caa07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:04:05.858798 32496 raft_consensus.cc:729] T 00000000000000000000000000000000 P efd74634030b45369f435df37f3caa07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efd74634030b45369f435df37f3caa07, State: Initialized, Role: FOLLOWER
I20260811 19:04:05.859324 32496 consensus_queue.cc:260] T 00000000000000000000000000000000 P efd74634030b45369f435df37f3caa07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efd74634030b45369f435df37f3caa07" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 35737 } }
I20260811 19:04:05.859495 32496 raft_consensus.cc:388] T 00000000000000000000000000000000 P efd74634030b45369f435df37f3caa07 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:04:05.859598 32496 raft_consensus.cc:482] T 00000000000000000000000000000000 P efd74634030b45369f435df37f3caa07 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:04:05.859722 32496 raft_consensus.cc:3037] T 00000000000000000000000000000000 P efd74634030b45369f435df37f3caa07 [term 0 FOLLOWER]: Advancing to term 1
Aug 11 19:04:05 dist-test-slave-0kls krb5kdc[32404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475045, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
I20260811 19:04:05.861162 32496 raft_consensus.cc:504] T 00000000000000000000000000000000 P efd74634030b45369f435df37f3caa07 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efd74634030b45369f435df37f3caa07" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 35737 } }
I20260811 19:04:05.861600 32496 leader_election.cc:302] T 00000000000000000000000000000000 P efd74634030b45369f435df37f3caa07 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: efd74634030b45369f435df37f3caa07; no voters:
I20260811 19:04:05.862071 32496 leader_election.cc:288] T 00000000000000000000000000000000 P efd74634030b45369f435df37f3caa07 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:04:05.862208 32501 raft_consensus.cc:2781] T 00000000000000000000000000000000 P efd74634030b45369f435df37f3caa07 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:04:05.862483 32501 raft_consensus.cc:686] T 00000000000000000000000000000000 P efd74634030b45369f435df37f3caa07 [term 1 LEADER]: Becoming Leader. State: Replica: efd74634030b45369f435df37f3caa07, State: Running, Role: LEADER
I20260811 19:04:05.862915 32501 consensus_queue.cc:237] T 00000000000000000000000000000000 P efd74634030b45369f435df37f3caa07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efd74634030b45369f435df37f3caa07" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 35737 } }
I20260811 19:04:05.863682 32496 sys_catalog.cc:564] T 00000000000000000000000000000000 P efd74634030b45369f435df37f3caa07 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:04:05.864820 32503 sys_catalog.cc:455] T 00000000000000000000000000000000 P efd74634030b45369f435df37f3caa07 [sys.catalog]: SysCatalogTable state changed. Reason: New leader efd74634030b45369f435df37f3caa07. Latest consensus state: current_term: 1 leader_uuid: "efd74634030b45369f435df37f3caa07" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efd74634030b45369f435df37f3caa07" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 35737 } } }
I20260811 19:04:05.864961 32503 sys_catalog.cc:458] T 00000000000000000000000000000000 P efd74634030b45369f435df37f3caa07 [sys.catalog]: This master's current role is: LEADER
I20260811 19:04:05.865447 32502 sys_catalog.cc:455] T 00000000000000000000000000000000 P efd74634030b45369f435df37f3caa07 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "efd74634030b45369f435df37f3caa07" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efd74634030b45369f435df37f3caa07" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 35737 } } }
I20260811 19:04:05.865643 32502 sys_catalog.cc:458] T 00000000000000000000000000000000 P efd74634030b45369f435df37f3caa07 [sys.catalog]: This master's current role is: LEADER
I20260811 19:04:05.867348 32499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:05.851488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54952 (local address 127.13.109.62:35737)
0811 19:04:05.852005 (+ 517us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:05.852024 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:05.852066 (+ 42us) server_negotiation.cc:407] Connection header received
0811 19:04:05.852874 (+ 808us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:05.852895 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:05.853247 (+ 352us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:05.853622 (+ 375us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:05.854617 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:05.855662 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:05.856630 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:05.857166 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:05.857373 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:05.860621 (+ 3248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:04:05.860627 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:04:05.860642 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:04:05.860676 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:04:05.863108 (+ 2432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:05.863744 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:05.863750 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:05.863756 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:05.863907 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:05.864277 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:05.864281 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:05.864283 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:05.864886 (+ 603us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:04:05.865117 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:05.866174 (+ 1057us) server_negotiation.cc:299] Negotiation successful
0811 19:04:05.866540 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":169,"threads_started":1}
W20260811 19:04:05.869323 32522 catalog_manager.cc:1558] T 00000000000000000000000000000000 P efd74634030b45369f435df37f3caa07: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 19:04:05.869405 32522 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 19:04:05.869472 32513 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:04:05.870302 32513 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:04:05.876286 32513 catalog_manager.cc:1347] Generated new cluster ID: a10f80a9f1b745dba97942fc64ae3c58
I20260811 19:04:05.876377 32513 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:04:05.884778 32513 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:04:05.886051 32513 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:04:05.905308 32513 catalog_manager.cc:5868] T 00000000000000000000000000000000 P efd74634030b45369f435df37f3caa07: Generated new TSK 0
I20260811 19:04:05.906320 32513 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.109.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.109.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.109.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.1.keytab.
Entry for principal kudu/127.13.109.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.1.keytab.
I20260811 19:04:05.953168 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.1:0
--local_ip_for_outbound_sockets=127.13.109.1
--webserver_interface=127.13.109.1
--webserver_port=0
--tserver_master_addrs=127.13.109.62:35737
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.1.keytab
--principal=kudu/127.13.109.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:37471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:04:06.094480 32527 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:04:06.094825 32527 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:04:06.094902 32527 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:04:06.099115 32527 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:04:06.099223 32527 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:04:06.099316 32527 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.1
I20260811 19:04:06.104993 32527 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:37471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.1.keytab
--principal=kudu/127.13.109.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.109.1
--webserver_port=0
--tserver_master_addrs=127.13.109.62:35737
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32527
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.1
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:04:06.106277 32527 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:04:06.107443 32527 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:04:06.118052 32533 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:04:06.117990 32532 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:04:06.117990 32535 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:04:06.118273 32527 server_base.cc:1034] running on GCE node
I20260811 19:04:06.118827 32527 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:04:06.119484 32527 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:04:06.120662 32527 hybrid_clock.cc:648] HybridClock initialized: now 1786475046120623 us; error 52 us; skew 500 ppm
Aug 11 19:04:06 dist-test-slave-0kls krb5kdc[32404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475046, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:04:06.123616 32527 init.cc:377] Logged in from keytab as kudu/127.13.109.1@KRBTEST.COM (short username kudu)
I20260811 19:04:06.124585 32527 webserver.cc:460] Webserver started at http://127.13.109.1:42349/ using document root <none> and password file <none>
I20260811 19:04:06.125209 32527 fs_manager.cc:359] Metadata directory not provided
I20260811 19:04:06.125311 32527 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:04:06.125564 32527 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:04:06.128580 32527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/minicluster-data/ts-0/data/instance:
uuid: "37a05a83014340ce9366abd8cbfa972b"
format_stamp: "Formatted at 2026-08-11 19:04:06 on dist-test-slave-0kls"
server_key: "253050937bd2618fc3a04c255e21c536"
server_key_iv: "596395cf21d55b2b2ddd808e4d494789"
server_key_version: "encryptionkey@0"
I20260811 19:04:06.129228 32527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/minicluster-data/ts-0/wal/instance:
uuid: "37a05a83014340ce9366abd8cbfa972b"
format_stamp: "Formatted at 2026-08-11 19:04:06 on dist-test-slave-0kls"
server_key: "253050937bd2618fc3a04c255e21c536"
server_key_iv: "596395cf21d55b2b2ddd808e4d494789"
server_key_version: "encryptionkey@0"
I20260811 19:04:06.133247 32527 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 19:04:06.135919 32542 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:04:06.137238 32527 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 19:04:06.137392 32527 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/minicluster-data/ts-0/wal
uuid: "37a05a83014340ce9366abd8cbfa972b"
format_stamp: "Formatted at 2026-08-11 19:04:06 on dist-test-slave-0kls"
server_key: "253050937bd2618fc3a04c255e21c536"
server_key_iv: "596395cf21d55b2b2ddd808e4d494789"
server_key_version: "encryptionkey@0"
I20260811 19:04:06.137523 32527 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:04:06.150666 32527 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:04:06.154609 32527 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:04:06.154846 32527 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:04:06.155539 32527 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:04:06.156531 32527 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:04:06.156613 32527 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:04:06.156688 32527 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:04:06.156742 32527 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:04:06.167209 32527 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.1:40391
I20260811 19:04:06.167237 32668 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.1:40391 every 8 connection(s)
I20260811 19:04:06.168226 32527 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
I20260811 19:04:06.172354 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 32527
I20260811 19:04:06.172472 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/minicluster-data/ts-0/wal/instance
I20260811 19:04:06.172811 13748 external_mini_cluster.cc:1428] Setting key 0f1a7ab951f84ba5e98a660f740bef1c
Aug 11 19:04:06 dist-test-slave-0kls krb5kdc[32404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475046, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.1@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.109.2@KRBTEST.COM; defaulting to no policy
I20260811 19:04:06.185525 32499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:06.170422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.1:58347 (local address 127.13.109.62:35737)
0811 19:04:06.170620 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:06.170627 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:06.171786 (+ 1159us) server_negotiation.cc:407] Connection header received
0811 19:04:06.172804 (+ 1018us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:06.172808 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:06.172880 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:06.172973 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:06.174803 (+ 1830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:06.175331 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:06.177184 (+ 1853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:06.177619 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:06.177756 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:06.180968 (+ 3212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:04:06.180972 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:04:06.180974 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:04:06.181005 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:04:06.182725 (+ 1720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:06.183492 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:06.183496 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:06.183497 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:06.183586 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:06.184075 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:06.184078 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:06.184080 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:06.184350 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:04:06.184458 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:06.185190 (+ 732us) server_negotiation.cc:299] Negotiation successful
0811 19:04:06.185336 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 19:04:06.186667 32672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:06.170870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:35737 (local address 127.13.109.1:58347)
0811 19:04:06.171602 (+ 732us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:06.171648 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 19:04:06.172569 (+ 921us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:04:06.173215 (+ 646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:04:06.173228 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:04:06.173875 (+ 647us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:04:06.174626 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:06.174651 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:06.175488 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:06.175495 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:06.177031 (+ 1536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:06.177044 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:06.177878 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:06.177883 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:06.178063 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:06.178786 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:04:06.178805 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:04:06.180709 (+ 1904us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:04:06.182886 (+ 2177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:04:06.182895 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:04:06.182916 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:04:06.183338 (+ 422us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:04:06.183720 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:04:06.183725 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:04:06.183730 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:04:06.183950 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:04:06.184474 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:04:06.184485 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:04:06.184904 (+ 419us) client_negotiation.cc:769] Sending connection context
0811 19:04:06.185106 (+ 202us) client_negotiation.cc:240] Negotiation successful
0811 19:04:06.185613 (+ 507us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":142,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.109.2@KRBTEST.COM" created.
I20260811 19:04:06.188468 32669 heartbeater.cc:347] Connected to a master server at 127.13.109.62:35737
I20260811 19:04:06.188793 32669 heartbeater.cc:464] Registering TS with master...
I20260811 19:04:06.189532 32669 heartbeater.cc:511] Master 127.13.109.62:35737 requested a full tablet report, sending...
I20260811 19:04:06.191543 32455 ts_manager.cc:194] Registered new tserver with Master: 37a05a83014340ce9366abd8cbfa972b (127.13.109.1:40391)
I20260811 19:04:06.192775 32455 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.109.1@KRBTEST.COM'} at 127.13.109.1:58347
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.109.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.2.keytab.
Entry for principal kudu/127.13.109.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.2.keytab.
I20260811 19:04:06.210620 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.2:0
--local_ip_for_outbound_sockets=127.13.109.2
--webserver_interface=127.13.109.2
--webserver_port=0
--tserver_master_addrs=127.13.109.62:35737
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.2.keytab
--principal=kudu/127.13.109.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:37471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:04:06.336107 32675 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:04:06.336532 32675 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:04:06.336634 32675 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:04:06.341233 32675 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:04:06.341347 32675 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:04:06.341450 32675 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.2
I20260811 19:04:06.347226 32675 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:37471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.2.keytab
--principal=kudu/127.13.109.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.109.2
--webserver_port=0
--tserver_master_addrs=127.13.109.62:35737
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32675
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.2
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:04:06.348523 32675 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:04:06.349632 32675 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:04:06.360234 32681 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:04:06.360234 32682 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:04:06.360512 32684 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:04:06.360526 32675 server_base.cc:1034] running on GCE node
I20260811 19:04:06.361133 32675 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:04:06.361900 32675 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:04:06.363090 32675 hybrid_clock.cc:648] HybridClock initialized: now 1786475046363075 us; error 30 us; skew 500 ppm
Aug 11 19:04:06 dist-test-slave-0kls krb5kdc[32404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475046, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:04:06.366101 32675 init.cc:377] Logged in from keytab as kudu/127.13.109.2@KRBTEST.COM (short username kudu)
I20260811 19:04:06.367172 32675 webserver.cc:460] Webserver started at http://127.13.109.2:44945/ using document root <none> and password file <none>
I20260811 19:04:06.367724 32675 fs_manager.cc:359] Metadata directory not provided
I20260811 19:04:06.367785 32675 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:04:06.367960 32675 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:04:06.370154 32675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/minicluster-data/ts-1/data/instance:
uuid: "daee41db60d54eeaac1fe01e830d6d78"
format_stamp: "Formatted at 2026-08-11 19:04:06 on dist-test-slave-0kls"
server_key: "a1e4d80933dc7818cb22fb0c7b686d15"
server_key_iv: "90297e2b54bc24c4ac77cf58dbe607c3"
server_key_version: "encryptionkey@0"
I20260811 19:04:06.370703 32675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/minicluster-data/ts-1/wal/instance:
uuid: "daee41db60d54eeaac1fe01e830d6d78"
format_stamp: "Formatted at 2026-08-11 19:04:06 on dist-test-slave-0kls"
server_key: "a1e4d80933dc7818cb22fb0c7b686d15"
server_key_iv: "90297e2b54bc24c4ac77cf58dbe607c3"
server_key_version: "encryptionkey@0"
I20260811 19:04:06.374651 32675 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 19:04:06.377275 32692 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:04:06.378814 32675 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 19:04:06.378948 32675 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/minicluster-data/ts-1/wal
uuid: "daee41db60d54eeaac1fe01e830d6d78"
format_stamp: "Formatted at 2026-08-11 19:04:06 on dist-test-slave-0kls"
server_key: "a1e4d80933dc7818cb22fb0c7b686d15"
server_key_iv: "90297e2b54bc24c4ac77cf58dbe607c3"
server_key_version: "encryptionkey@0"
I20260811 19:04:06.379063 32675 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:04:06.401298 32675 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:04:06.404754 32675 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:04:06.404999 32675 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:04:06.405735 32675 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:04:06.406666 32675 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:04:06.406714 32675 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:04:06.406790 32675 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:04:06.406822 32675 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:04:06.416244 32675 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.2:39925
I20260811 19:04:06.416285 352 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.2:39925 every 8 connection(s)
I20260811 19:04:06.417156 32675 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
I20260811 19:04:06.419364 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 32675
I20260811 19:04:06.419502 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/minicluster-data/ts-1/wal/instance
I20260811 19:04:06.419829 13748 external_mini_cluster.cc:1428] Setting key 8bcef22319f65232e108d1265142473f
Aug 11 19:04:06 dist-test-slave-0kls krb5kdc[32404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475046, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.2@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.109.3@KRBTEST.COM; defaulting to no policy
I20260811 19:04:06.433331 32499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:06.419087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.2:37709 (local address 127.13.109.62:35737)
0811 19:04:06.419235 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:06.419240 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:06.420234 (+ 994us) server_negotiation.cc:407] Connection header received
0811 19:04:06.421700 (+ 1466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:06.421702 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:06.421745 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:06.421891 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:06.423506 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:06.424381 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:06.425565 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:06.426011 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:06.426137 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:06.428953 (+ 2816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:04:06.428959 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:04:06.428963 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:04:06.428993 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:04:06.430849 (+ 1856us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:06.431420 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:06.431426 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:06.431429 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:06.431526 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:06.431904 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:06.431907 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:06.431908 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:06.432291 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:04:06.432379 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:06.433062 (+ 683us) server_negotiation.cc:299] Negotiation successful
0811 19:04:06.433201 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 19:04:06.434432 355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:06.419503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:35737 (local address 127.13.109.2:37709)
0811 19:04:06.420048 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:06.420102 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 19:04:06.421464 (+ 1362us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:04:06.422075 (+ 611us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:04:06.422086 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:04:06.422647 (+ 561us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:04:06.423343 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:06.423364 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:06.424546 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:06.424549 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:06.425431 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:06.425441 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:06.426180 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:06.426184 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:06.426329 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:06.427003 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:04:06.427023 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:04:06.428752 (+ 1729us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:04:06.431013 (+ 2261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:04:06.431019 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:04:06.431033 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:04:06.431297 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:04:06.431637 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:04:06.431640 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:04:06.431641 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:04:06.431804 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:04:06.432422 (+ 618us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:04:06.432427 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:04:06.432814 (+ 387us) client_negotiation.cc:769] Sending connection context
0811 19:04:06.432995 (+ 181us) client_negotiation.cc:240] Negotiation successful
0811 19:04:06.433286 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":154,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.109.3@KRBTEST.COM" created.
I20260811 19:04:06.435791 353 heartbeater.cc:347] Connected to a master server at 127.13.109.62:35737
I20260811 19:04:06.436038 353 heartbeater.cc:464] Registering TS with master...
I20260811 19:04:06.436573 353 heartbeater.cc:511] Master 127.13.109.62:35737 requested a full tablet report, sending...
I20260811 19:04:06.437827 32455 ts_manager.cc:194] Registered new tserver with Master: daee41db60d54eeaac1fe01e830d6d78 (127.13.109.2:39925)
I20260811 19:04:06.438509 32455 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.109.2@KRBTEST.COM'} at 127.13.109.2:37709
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.109.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.3.keytab.
Entry for principal kudu/127.13.109.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.3.keytab.
I20260811 19:04:06.454005 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.3:0
--local_ip_for_outbound_sockets=127.13.109.3
--webserver_interface=127.13.109.3
--webserver_port=0
--tserver_master_addrs=127.13.109.62:35737
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.3.keytab
--principal=kudu/127.13.109.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:37471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:04:06.575623 359 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:04:06.575892 359 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:04:06.575937 359 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:04:06.580086 359 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:04:06.580166 359 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:04:06.580281 359 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.3
I20260811 19:04:06.587533 359 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:37471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.3.keytab
--principal=kudu/127.13.109.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.109.3
--webserver_port=0
--tserver_master_addrs=127.13.109.62:35737
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.359
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.3
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:04:06.588932 359 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:04:06.590160 359 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:04:06.600742 368 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:04:06.600739 365 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:04:06.600759 366 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:04:06.601092 359 server_base.cc:1034] running on GCE node
I20260811 19:04:06.601732 359 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:04:06.602491 359 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:04:06.603698 359 hybrid_clock.cc:648] HybridClock initialized: now 1786475046603673 us; error 37 us; skew 500 ppm
Aug 11 19:04:06 dist-test-slave-0kls krb5kdc[32404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475046, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:04:06.606714 359 init.cc:377] Logged in from keytab as kudu/127.13.109.3@KRBTEST.COM (short username kudu)
I20260811 19:04:06.607784 359 webserver.cc:460] Webserver started at http://127.13.109.3:37209/ using document root <none> and password file <none>
I20260811 19:04:06.608448 359 fs_manager.cc:359] Metadata directory not provided
I20260811 19:04:06.608516 359 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:04:06.608702 359 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:04:06.611060 359 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/minicluster-data/ts-2/data/instance:
uuid: "a5f97462e246464da817a89feff3d0f1"
format_stamp: "Formatted at 2026-08-11 19:04:06 on dist-test-slave-0kls"
server_key: "56cd5a4a67301eebdfcc90c020ab3cb4"
server_key_iv: "95d643612dbc11f6b6b4ac7aa9cb9e66"
server_key_version: "encryptionkey@0"
I20260811 19:04:06.611655 359 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/minicluster-data/ts-2/wal/instance:
uuid: "a5f97462e246464da817a89feff3d0f1"
format_stamp: "Formatted at 2026-08-11 19:04:06 on dist-test-slave-0kls"
server_key: "56cd5a4a67301eebdfcc90c020ab3cb4"
server_key_iv: "95d643612dbc11f6b6b4ac7aa9cb9e66"
server_key_version: "encryptionkey@0"
I20260811 19:04:06.615850 359 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 19:04:06.618624 375 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:04:06.620018 359 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 19:04:06.620199 359 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/minicluster-data/ts-2/wal
uuid: "a5f97462e246464da817a89feff3d0f1"
format_stamp: "Formatted at 2026-08-11 19:04:06 on dist-test-slave-0kls"
server_key: "56cd5a4a67301eebdfcc90c020ab3cb4"
server_key_iv: "95d643612dbc11f6b6b4ac7aa9cb9e66"
server_key_version: "encryptionkey@0"
I20260811 19:04:06.620327 359 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:04:06.659322 359 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:04:06.663082 359 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:04:06.663326 359 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:04:06.664062 359 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:04:06.665087 359 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:04:06.665145 359 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:04:06.665218 359 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:04:06.665254 359 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:04:06.675352 359 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.3:37263
I20260811 19:04:06.675374 501 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.3:37263 every 8 connection(s)
I20260811 19:04:06.676357 359 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
I20260811 19:04:06.683314 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 359
I20260811 19:04:06.683470 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/minicluster-data/ts-2/wal/instance
I20260811 19:04:06.683858 13748 external_mini_cluster.cc:1428] Setting key 7ce770604d1a34c1f5e6baea0a81169e
Aug 11 19:04:06 dist-test-slave-0kls krb5kdc[32404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475046, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.3@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
I20260811 19:04:06.692718 32499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:06.678398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.3:42153 (local address 127.13.109.62:35737)
0811 19:04:06.678554 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:06.678559 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:06.679523 (+ 964us) server_negotiation.cc:407] Connection header received
0811 19:04:06.681180 (+ 1657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:06.681183 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:06.681254 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:06.681369 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:06.682747 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:06.683622 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:06.685225 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:06.685548 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:06.685651 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:06.688709 (+ 3058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:04:06.688713 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:04:06.688715 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:04:06.688771 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:04:06.690592 (+ 1821us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:06.691191 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:06.691196 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:06.691199 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:06.691294 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:06.691670 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:06.691672 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:06.691673 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:06.691878 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:04:06.691959 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:06.692465 (+ 506us) server_negotiation.cc:299] Negotiation successful
0811 19:04:06.692575 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 19:04:06.693599 504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:06.678784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:35737 (local address 127.13.109.3:42153)
0811 19:04:06.679308 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:06.679359 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 19:04:06.680775 (+ 1416us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:04:06.681575 (+ 800us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:04:06.681584 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:04:06.682004 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:04:06.682560 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:06.682583 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:06.683797 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:06.683800 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:06.685074 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:06.685085 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:06.685867 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:06.685871 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:06.686077 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:06.686782 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:04:06.686800 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:04:06.688461 (+ 1661us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:04:06.690720 (+ 2259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:04:06.690726 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:04:06.690742 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:04:06.691018 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:04:06.691421 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:04:06.691424 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:04:06.691425 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:04:06.691572 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:04:06.691976 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:04:06.691981 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:04:06.692242 (+ 261us) client_negotiation.cc:769] Sending connection context
0811 19:04:06.692411 (+ 169us) client_negotiation.cc:240] Negotiation successful
0811 19:04:06.692698 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":113,"threads_started":1}
I20260811 19:04:06.695009 502 heartbeater.cc:347] Connected to a master server at 127.13.109.62:35737
I20260811 19:04:06.695308 502 heartbeater.cc:464] Registering TS with master...
I20260811 19:04:06.695865 502 heartbeater.cc:511] Master 127.13.109.62:35737 requested a full tablet report, sending...
I20260811 19:04:06.697016 32455 ts_manager.cc:194] Registered new tserver with Master: a5f97462e246464da817a89feff3d0f1 (127.13.109.3:37263)
I20260811 19:04:06.697640 32455 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.109.3@KRBTEST.COM'} at 127.13.109.3:42153
I20260811 19:04:06.698241 13748 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 19:04:06.707868 32499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:06.700706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54960 (local address 127.13.109.62:35737)
0811 19:04:06.700841 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:06.700846 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:06.701001 (+ 155us) server_negotiation.cc:407] Connection header received
0811 19:04:06.701105 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:06.701107 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:06.701143 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:06.701219 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:06.701971 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:06.702499 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:06.703307 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:06.703512 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:06.703587 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:06.704885 (+ 1298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:04:06.704888 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:04:06.704890 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:04:06.704913 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:04:06.706407 (+ 1494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:06.706800 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:06.706803 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:06.706804 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:06.706878 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:06.707106 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:06.707108 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:06.707109 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:06.707345 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:04:06.707449 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:06.707648 (+ 199us) server_negotiation.cc:299] Negotiation successful
0811 19:04:06.707740 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 19:04:06.711134 32454 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54960:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 19:04:06.714228 32454 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 19:04:06.730041 521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:06.724268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.2:39925 (local address 127.0.0.1:55358)
0811 19:04:06.724514 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:06.724529 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 19:04:06.724728 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:04:06.725341 (+ 613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:04:06.725344 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:04:06.725361 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:04:06.725458 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:06.725469 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:06.727162 (+ 1693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:06.727166 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:06.728062 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:06.728072 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:06.729414 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:06.729423 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:06.729578 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:06.729641 (+ 63us) client_negotiation.cc:769] Sending connection context
0811 19:04:06.729731 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 19:04:06.729883 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":106,"threads_started":1}
I20260811 19:04:06.730619 523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:06.724618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.1:40391 (local address 127.0.0.1:39762)
0811 19:04:06.724923 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:06.724937 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 19:04:06.725018 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:04:06.725826 (+ 808us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:04:06.725829 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:04:06.725847 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:04:06.725962 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:06.725969 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:06.727332 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:06.727336 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:06.728519 (+ 1183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:06.728534 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:06.729689 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:06.729694 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:06.730042 (+ 348us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:06.730194 (+ 152us) client_negotiation.cc:769] Sending connection context
0811 19:04:06.730269 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 19:04:06.730436 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"mutex_wait_us":200,"thread_start_us":93,"threads_started":1}
I20260811 19:04:06.730619 519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:06.723502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.3:37263 (local address 127.0.0.1:55680)
0811 19:04:06.723886 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:06.723946 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 19:04:06.724133 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:04:06.724880 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:04:06.724886 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:04:06.724915 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:04:06.725071 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:06.725079 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:06.726984 (+ 1905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:06.726987 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:06.728561 (+ 1574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:06.728573 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:06.729689 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:06.729694 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:06.730045 (+ 351us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:06.730194 (+ 149us) client_negotiation.cc:769] Sending connection context
0811 19:04:06.730269 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 19:04:06.730436 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"mutex_wait_us":309,"thread_start_us":81,"threads_started":1}
I20260811 19:04:06.731014 522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:06.724421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55358 (local address 127.13.109.2:39925)
0811 19:04:06.724852 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:06.724866 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:06.724887 (+ 21us) server_negotiation.cc:407] Connection header received
0811 19:04:06.724958 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:06.724963 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:06.725189 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:06.725610 (+ 421us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:04:06.725775 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:06.726984 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:06.728255 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:06.729265 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:06.730437 (+ 1172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:06.730604 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:06.730680 (+ 76us) server_negotiation.cc:299] Negotiation successful
0811 19:04:06.730785 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":104,"threads_started":1}
I20260811 19:04:06.731139 520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:06.723680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55680 (local address 127.13.109.3:37263)
0811 19:04:06.724302 (+ 622us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:06.724317 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:06.724344 (+ 27us) server_negotiation.cc:407] Connection header received
0811 19:04:06.724424 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:06.724430 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:06.724687 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:06.725266 (+ 579us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:04:06.725437 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:06.726797 (+ 1360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:06.728701 (+ 1904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:06.729524 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:06.730466 (+ 942us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:06.730805 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:06.730876 (+ 71us) server_negotiation.cc:299] Negotiation successful
0811 19:04:06.730978 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":508,"thread_start_us":187,"threads_started":1}
I20260811 19:04:06.731885 524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:06.725076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39762 (local address 127.13.109.1:40391)
0811 19:04:06.725376 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:06.725382 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:06.725404 (+ 22us) server_negotiation.cc:407] Connection header received
0811 19:04:06.725501 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:06.725511 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:06.725643 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:06.725832 (+ 189us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:04:06.726091 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:06.727195 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:06.728688 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:06.729524 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:06.731416 (+ 1892us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:06.731581 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:06.731635 (+ 54us) server_negotiation.cc:299] Negotiation successful
0811 19:04:06.731728 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":132,"threads_started":1}
I20260811 19:04:06.734333 32594 tablet_service.cc:1467] Processing CreateTablet for tablet 79d29cb7f2b64025be52c9774b821567 (DEFAULT_TABLE table=test-table [id=07c33e5257ab4852b0d1fe0bbd9cfc93]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:04:06.734223 428 tablet_service.cc:1467] Processing CreateTablet for tablet 79d29cb7f2b64025be52c9774b821567 (DEFAULT_TABLE table=test-table [id=07c33e5257ab4852b0d1fe0bbd9cfc93]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:04:06.734227 32744 tablet_service.cc:1467] Processing CreateTablet for tablet 79d29cb7f2b64025be52c9774b821567 (DEFAULT_TABLE table=test-table [id=07c33e5257ab4852b0d1fe0bbd9cfc93]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:04:06.735453 32744 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79d29cb7f2b64025be52c9774b821567. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:04:06.735453 428 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79d29cb7f2b64025be52c9774b821567. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:04:06.735450 32594 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79d29cb7f2b64025be52c9774b821567. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:04:06.743141 525 tablet_bootstrap.cc:492] T 79d29cb7f2b64025be52c9774b821567 P 37a05a83014340ce9366abd8cbfa972b: Bootstrap starting.
I20260811 19:04:06.743889 527 tablet_bootstrap.cc:492] T 79d29cb7f2b64025be52c9774b821567 P a5f97462e246464da817a89feff3d0f1: Bootstrap starting.
I20260811 19:04:06.743937 528 tablet_bootstrap.cc:492] T 79d29cb7f2b64025be52c9774b821567 P daee41db60d54eeaac1fe01e830d6d78: Bootstrap starting.
I20260811 19:04:06.746481 527 tablet_bootstrap.cc:654] T 79d29cb7f2b64025be52c9774b821567 P a5f97462e246464da817a89feff3d0f1: Neither blocks nor log segments found. Creating new log.
I20260811 19:04:06.746974 528 tablet_bootstrap.cc:654] T 79d29cb7f2b64025be52c9774b821567 P daee41db60d54eeaac1fe01e830d6d78: Neither blocks nor log segments found. Creating new log.
I20260811 19:04:06.748035 525 tablet_bootstrap.cc:654] T 79d29cb7f2b64025be52c9774b821567 P 37a05a83014340ce9366abd8cbfa972b: Neither blocks nor log segments found. Creating new log.
I20260811 19:04:06.748083 528 log.cc:824] T 79d29cb7f2b64025be52c9774b821567 P daee41db60d54eeaac1fe01e830d6d78: Log is configured to *not* fsync() on all Append() calls
I20260811 19:04:06.748535 527 log.cc:824] T 79d29cb7f2b64025be52c9774b821567 P a5f97462e246464da817a89feff3d0f1: Log is configured to *not* fsync() on all Append() calls
I20260811 19:04:06.748839 525 log.cc:824] T 79d29cb7f2b64025be52c9774b821567 P 37a05a83014340ce9366abd8cbfa972b: Log is configured to *not* fsync() on all Append() calls
I20260811 19:04:06.750597 527 tablet_bootstrap.cc:492] T 79d29cb7f2b64025be52c9774b821567 P a5f97462e246464da817a89feff3d0f1: No bootstrap required, opened a new log
I20260811 19:04:06.750597 528 tablet_bootstrap.cc:492] T 79d29cb7f2b64025be52c9774b821567 P daee41db60d54eeaac1fe01e830d6d78: No bootstrap required, opened a new log
I20260811 19:04:06.750831 527 ts_tablet_manager.cc:1397] T 79d29cb7f2b64025be52c9774b821567 P a5f97462e246464da817a89feff3d0f1: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 19:04:06.750833 528 ts_tablet_manager.cc:1397] T 79d29cb7f2b64025be52c9774b821567 P daee41db60d54eeaac1fe01e830d6d78: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 19:04:06.750938 525 tablet_bootstrap.cc:492] T 79d29cb7f2b64025be52c9774b821567 P 37a05a83014340ce9366abd8cbfa972b: No bootstrap required, opened a new log
I20260811 19:04:06.751116 525 ts_tablet_manager.cc:1397] T 79d29cb7f2b64025be52c9774b821567 P 37a05a83014340ce9366abd8cbfa972b: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260811 19:04:06.754339 527 raft_consensus.cc:348] T 79d29cb7f2b64025be52c9774b821567 P a5f97462e246464da817a89feff3d0f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f97462e246464da817a89feff3d0f1" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 37263 } } peers { permanent_uuid: "daee41db60d54eeaac1fe01e830d6d78" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 39925 } } peers { permanent_uuid: "37a05a83014340ce9366abd8cbfa972b" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 40391 } }
I20260811 19:04:06.754784 527 raft_consensus.cc:374] T 79d29cb7f2b64025be52c9774b821567 P a5f97462e246464da817a89feff3d0f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:04:06.754901 527 raft_consensus.cc:729] T 79d29cb7f2b64025be52c9774b821567 P a5f97462e246464da817a89feff3d0f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5f97462e246464da817a89feff3d0f1, State: Initialized, Role: FOLLOWER
I20260811 19:04:06.754956 528 raft_consensus.cc:348] T 79d29cb7f2b64025be52c9774b821567 P daee41db60d54eeaac1fe01e830d6d78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f97462e246464da817a89feff3d0f1" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 37263 } } peers { permanent_uuid: "daee41db60d54eeaac1fe01e830d6d78" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 39925 } } peers { permanent_uuid: "37a05a83014340ce9366abd8cbfa972b" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 40391 } }
I20260811 19:04:06.755309 528 raft_consensus.cc:374] T 79d29cb7f2b64025be52c9774b821567 P daee41db60d54eeaac1fe01e830d6d78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:04:06.755391 528 raft_consensus.cc:729] T 79d29cb7f2b64025be52c9774b821567 P daee41db60d54eeaac1fe01e830d6d78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: daee41db60d54eeaac1fe01e830d6d78, State: Initialized, Role: FOLLOWER
I20260811 19:04:06.755560 527 consensus_queue.cc:260] T 79d29cb7f2b64025be52c9774b821567 P a5f97462e246464da817a89feff3d0f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f97462e246464da817a89feff3d0f1" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 37263 } } peers { permanent_uuid: "daee41db60d54eeaac1fe01e830d6d78" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 39925 } } peers { permanent_uuid: "37a05a83014340ce9366abd8cbfa972b" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 40391 } }
I20260811 19:04:06.755570 525 raft_consensus.cc:348] T 79d29cb7f2b64025be52c9774b821567 P 37a05a83014340ce9366abd8cbfa972b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f97462e246464da817a89feff3d0f1" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 37263 } } peers { permanent_uuid: "daee41db60d54eeaac1fe01e830d6d78" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 39925 } } peers { permanent_uuid: "37a05a83014340ce9366abd8cbfa972b" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 40391 } }
I20260811 19:04:06.755852 525 raft_consensus.cc:374] T 79d29cb7f2b64025be52c9774b821567 P 37a05a83014340ce9366abd8cbfa972b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:04:06.755921 525 raft_consensus.cc:729] T 79d29cb7f2b64025be52c9774b821567 P 37a05a83014340ce9366abd8cbfa972b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37a05a83014340ce9366abd8cbfa972b, State: Initialized, Role: FOLLOWER
I20260811 19:04:06.755852 528 consensus_queue.cc:260] T 79d29cb7f2b64025be52c9774b821567 P daee41db60d54eeaac1fe01e830d6d78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f97462e246464da817a89feff3d0f1" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 37263 } } peers { permanent_uuid: "daee41db60d54eeaac1fe01e830d6d78" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 39925 } } peers { permanent_uuid: "37a05a83014340ce9366abd8cbfa972b" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 40391 } }
I20260811 19:04:06.756378 525 consensus_queue.cc:260] T 79d29cb7f2b64025be52c9774b821567 P 37a05a83014340ce9366abd8cbfa972b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f97462e246464da817a89feff3d0f1" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 37263 } } peers { permanent_uuid: "daee41db60d54eeaac1fe01e830d6d78" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 39925 } } peers { permanent_uuid: "37a05a83014340ce9366abd8cbfa972b" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 40391 } }
I20260811 19:04:06.756774 527 ts_tablet_manager.cc:1428] T 79d29cb7f2b64025be52c9774b821567 P a5f97462e246464da817a89feff3d0f1: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 19:04:06.756942 528 ts_tablet_manager.cc:1428] T 79d29cb7f2b64025be52c9774b821567 P daee41db60d54eeaac1fe01e830d6d78: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 19:04:06.757082 32669 heartbeater.cc:503] Master 127.13.109.62:35737 was elected leader, sending a full tablet report...
I20260811 19:04:06.757119 502 heartbeater.cc:503] Master 127.13.109.62:35737 was elected leader, sending a full tablet report...
I20260811 19:04:06.757282 525 ts_tablet_manager.cc:1428] T 79d29cb7f2b64025be52c9774b821567 P 37a05a83014340ce9366abd8cbfa972b: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 19:04:06.757386 353 heartbeater.cc:503] Master 127.13.109.62:35737 was elected leader, sending a full tablet report...
W20260811 19:04:06.918365 354 tablet.cc:2369] T 79d29cb7f2b64025be52c9774b821567 P daee41db60d54eeaac1fe01e830d6d78: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 19:04:06.919677 32670 tablet.cc:2369] T 79d29cb7f2b64025be52c9774b821567 P 37a05a83014340ce9366abd8cbfa972b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 19:04:06.927412 503 tablet.cc:2369] T 79d29cb7f2b64025be52c9774b821567 P a5f97462e246464da817a89feff3d0f1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 19:04:06.943822 534 raft_consensus.cc:482] T 79d29cb7f2b64025be52c9774b821567 P 37a05a83014340ce9366abd8cbfa972b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:04:06.944056 534 raft_consensus.cc:504] T 79d29cb7f2b64025be52c9774b821567 P 37a05a83014340ce9366abd8cbfa972b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f97462e246464da817a89feff3d0f1" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 37263 } } peers { permanent_uuid: "daee41db60d54eeaac1fe01e830d6d78" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 39925 } } peers { permanent_uuid: "37a05a83014340ce9366abd8cbfa972b" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 40391 } }
I20260811 19:04:06.945232 534 leader_election.cc:288] T 79d29cb7f2b64025be52c9774b821567 P 37a05a83014340ce9366abd8cbfa972b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a5f97462e246464da817a89feff3d0f1 (127.13.109.3:37263), daee41db60d54eeaac1fe01e830d6d78 (127.13.109.2:39925)
I20260811 19:04:06.951086 520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:06.945709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.1:35005 (local address 127.13.109.3:37263)
0811 19:04:06.945929 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:06.945933 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:06.946100 (+ 167us) server_negotiation.cc:407] Connection header received
0811 19:04:06.946257 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:06.946261 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:06.946316 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:06.946416 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:04:06.946861 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:06.947664 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:06.949733 (+ 2069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:06.950396 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:06.950536 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:06.950608 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:06.950871 (+ 263us) server_negotiation.cc:299] Negotiation successful
0811 19:04:06.950944 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118}
I20260811 19:04:06.951114 522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:06.946230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.1:60241 (local address 127.13.109.2:39925)
0811 19:04:06.946367 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:06.946373 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:06.946556 (+ 183us) server_negotiation.cc:407] Connection header received
0811 19:04:06.946654 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:06.946657 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:06.946824 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:06.946901 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:04:06.947176 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:06.947855 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:06.949733 (+ 1878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:06.950397 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:06.950536 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:06.950595 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:06.950871 (+ 276us) server_negotiation.cc:299] Negotiation successful
0811 19:04:06.950949 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 19:04:06.951195 536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:06.946089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.2:39925 (local address 127.13.109.1:60241)
0811 19:04:06.946452 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:06.946465 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 19:04:06.946563 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:04:06.946925 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:04:06.946927 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:04:06.946944 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:04:06.947039 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:06.947044 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:06.947980 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:06.947983 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:06.949560 (+ 1577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:06.949572 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:06.950548 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:06.950551 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:06.950703 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:06.950751 (+ 48us) client_negotiation.cc:769] Sending connection context
0811 19:04:06.950832 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 19:04:06.950997 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"mutex_wait_us":292,"thread_start_us":108,"threads_started":1}
I20260811 19:04:06.951200 535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:06.945611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.3:37263 (local address 127.13.109.1:35005)
0811 19:04:06.945988 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:06.946008 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 19:04:06.946136 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:04:06.946451 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:04:06.946454 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:04:06.946480 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:04:06.946641 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:06.946648 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:06.947880 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:06.947883 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:06.949560 (+ 1677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:06.949572 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:06.950537 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:06.950543 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:06.950669 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:06.950747 (+ 78us) client_negotiation.cc:769] Sending connection context
0811 19:04:06.950832 (+ 85us) client_negotiation.cc:240] Negotiation successful
0811 19:04:06.950997 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"mutex_wait_us":165,"thread_start_us":117,"threads_started":1}
I20260811 19:04:06.951916 32765 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79d29cb7f2b64025be52c9774b821567" candidate_uuid: "37a05a83014340ce9366abd8cbfa972b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "daee41db60d54eeaac1fe01e830d6d78" is_pre_election: true
I20260811 19:04:06.951946 450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79d29cb7f2b64025be52c9774b821567" candidate_uuid: "37a05a83014340ce9366abd8cbfa972b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5f97462e246464da817a89feff3d0f1" is_pre_election: true
I20260811 19:04:06.952255 450 raft_consensus.cc:2454] T 79d29cb7f2b64025be52c9774b821567 P a5f97462e246464da817a89feff3d0f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 37a05a83014340ce9366abd8cbfa972b in term 0.
I20260811 19:04:06.952258 32765 raft_consensus.cc:2454] T 79d29cb7f2b64025be52c9774b821567 P daee41db60d54eeaac1fe01e830d6d78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 37a05a83014340ce9366abd8cbfa972b in term 0.
I20260811 19:04:06.952807 32543 leader_election.cc:302] T 79d29cb7f2b64025be52c9774b821567 P 37a05a83014340ce9366abd8cbfa972b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 37a05a83014340ce9366abd8cbfa972b, daee41db60d54eeaac1fe01e830d6d78; no voters:
I20260811 19:04:06.953125 534 raft_consensus.cc:2781] T 79d29cb7f2b64025be52c9774b821567 P 37a05a83014340ce9366abd8cbfa972b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 19:04:06.953243 534 raft_consensus.cc:482] T 79d29cb7f2b64025be52c9774b821567 P 37a05a83014340ce9366abd8cbfa972b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 19:04:06.953325 534 raft_consensus.cc:3037] T 79d29cb7f2b64025be52c9774b821567 P 37a05a83014340ce9366abd8cbfa972b [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:04:06.954536 534 raft_consensus.cc:504] T 79d29cb7f2b64025be52c9774b821567 P 37a05a83014340ce9366abd8cbfa972b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f97462e246464da817a89feff3d0f1" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 37263 } } peers { permanent_uuid: "daee41db60d54eeaac1fe01e830d6d78" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 39925 } } peers { permanent_uuid: "37a05a83014340ce9366abd8cbfa972b" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 40391 } }
I20260811 19:04:06.954931 534 leader_election.cc:288] T 79d29cb7f2b64025be52c9774b821567 P 37a05a83014340ce9366abd8cbfa972b [CANDIDATE]: Term 1 election: Requested vote from peers a5f97462e246464da817a89feff3d0f1 (127.13.109.3:37263), daee41db60d54eeaac1fe01e830d6d78 (127.13.109.2:39925)
I20260811 19:04:06.955511 450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79d29cb7f2b64025be52c9774b821567" candidate_uuid: "37a05a83014340ce9366abd8cbfa972b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5f97462e246464da817a89feff3d0f1"
I20260811 19:04:06.955582 32765 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79d29cb7f2b64025be52c9774b821567" candidate_uuid: "37a05a83014340ce9366abd8cbfa972b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "daee41db60d54eeaac1fe01e830d6d78"
I20260811 19:04:06.955720 450 raft_consensus.cc:3037] T 79d29cb7f2b64025be52c9774b821567 P a5f97462e246464da817a89feff3d0f1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:04:06.955735 32765 raft_consensus.cc:3037] T 79d29cb7f2b64025be52c9774b821567 P daee41db60d54eeaac1fe01e830d6d78 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:04:06.956856 450 raft_consensus.cc:2454] T 79d29cb7f2b64025be52c9774b821567 P a5f97462e246464da817a89feff3d0f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 37a05a83014340ce9366abd8cbfa972b in term 1.
I20260811 19:04:06.956923 32765 raft_consensus.cc:2454] T 79d29cb7f2b64025be52c9774b821567 P daee41db60d54eeaac1fe01e830d6d78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 37a05a83014340ce9366abd8cbfa972b in term 1.
I20260811 19:04:06.957237 32543 leader_election.cc:302] T 79d29cb7f2b64025be52c9774b821567 P 37a05a83014340ce9366abd8cbfa972b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 37a05a83014340ce9366abd8cbfa972b, a5f97462e246464da817a89feff3d0f1; no voters:
I20260811 19:04:06.957458 534 raft_consensus.cc:2781] T 79d29cb7f2b64025be52c9774b821567 P 37a05a83014340ce9366abd8cbfa972b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:04:06.957726 534 raft_consensus.cc:686] T 79d29cb7f2b64025be52c9774b821567 P 37a05a83014340ce9366abd8cbfa972b [term 1 LEADER]: Becoming Leader. State: Replica: 37a05a83014340ce9366abd8cbfa972b, State: Running, Role: LEADER
I20260811 19:04:06.958117 534 consensus_queue.cc:237] T 79d29cb7f2b64025be52c9774b821567 P 37a05a83014340ce9366abd8cbfa972b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f97462e246464da817a89feff3d0f1" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 37263 } } peers { permanent_uuid: "daee41db60d54eeaac1fe01e830d6d78" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 39925 } } peers { permanent_uuid: "37a05a83014340ce9366abd8cbfa972b" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 40391 } }
I20260811 19:04:06.961845 32454 catalog_manager.cc:5495] T 79d29cb7f2b64025be52c9774b821567 P 37a05a83014340ce9366abd8cbfa972b reported cstate change: term changed from 0 to 1, leader changed from <none> to 37a05a83014340ce9366abd8cbfa972b (127.13.109.1). New cstate: current_term: 1 leader_uuid: "37a05a83014340ce9366abd8cbfa972b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f97462e246464da817a89feff3d0f1" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 37263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "daee41db60d54eeaac1fe01e830d6d78" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 39925 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "37a05a83014340ce9366abd8cbfa972b" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 40391 } health_report { overall_health: HEALTHY } } }
I20260811 19:04:07.026629 524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:07.022533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39764 (local address 127.13.109.1:40391)
0811 19:04:07.022761 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:07.022766 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:07.022854 (+ 88us) server_negotiation.cc:407] Connection header received
0811 19:04:07.023062 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:07.023064 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:07.023126 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:07.023264 (+ 138us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:04:07.023564 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:07.024080 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:07.025011 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:07.025237 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:07.025358 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:07.025702 (+ 344us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:04:07.026195 (+ 493us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:04:07.026340 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:07.026405 (+ 65us) server_negotiation.cc:299] Negotiation successful
0811 19:04:07.026471 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123}
I20260811 19:04:07.033825 32765 raft_consensus.cc:1261] T 79d29cb7f2b64025be52c9774b821567 P daee41db60d54eeaac1fe01e830d6d78 [term 1 FOLLOWER]: Refusing update from remote peer 37a05a83014340ce9366abd8cbfa972b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:04:07.033825 450 raft_consensus.cc:1261] T 79d29cb7f2b64025be52c9774b821567 P a5f97462e246464da817a89feff3d0f1 [term 1 FOLLOWER]: Refusing update from remote peer 37a05a83014340ce9366abd8cbfa972b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:04:07.034718 537 consensus_queue.cc:1035] T 79d29cb7f2b64025be52c9774b821567 P 37a05a83014340ce9366abd8cbfa972b [LEADER]: Connected to new peer: Peer: permanent_uuid: "daee41db60d54eeaac1fe01e830d6d78" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 39925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:04:07.034950 534 consensus_queue.cc:1035] T 79d29cb7f2b64025be52c9774b821567 P 37a05a83014340ce9366abd8cbfa972b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5f97462e246464da817a89feff3d0f1" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 37263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:04:07.053870 32455 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54960:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:04:07.054097 32455 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54960:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:04:07.056869 32455 catalog_manager.cc:5782] T 00000000000000000000000000000000 P efd74634030b45369f435df37f3caa07: Sending DeleteTablet for 3 replicas of tablet 79d29cb7f2b64025be52c9774b821567
I20260811 19:04:07.057888 428 tablet_service.cc:1514] Processing DeleteTablet for tablet 79d29cb7f2b64025be52c9774b821567 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:04:07 UTC) from {username='kudu', principal='kudu/127.13.109.62@KRBTEST.COM'} at 127.0.0.1:55680
I20260811 19:04:07.057950 32744 tablet_service.cc:1514] Processing DeleteTablet for tablet 79d29cb7f2b64025be52c9774b821567 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:04:07 UTC) from {username='kudu', principal='kudu/127.13.109.62@KRBTEST.COM'} at 127.0.0.1:55358
I20260811 19:04:07.058516 554 tablet_replica.cc:330] T 79d29cb7f2b64025be52c9774b821567 P daee41db60d54eeaac1fe01e830d6d78: stopping tablet replica
I20260811 19:04:07.058542 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 32527
I20260811 19:04:07.060989 553 tablet_replica.cc:330] T 79d29cb7f2b64025be52c9774b821567 P a5f97462e246464da817a89feff3d0f1: stopping tablet replica
I20260811 19:04:07.061502 554 raft_consensus.cc:2229] T 79d29cb7f2b64025be52c9774b821567 P daee41db60d54eeaac1fe01e830d6d78 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:04:07.061502 553 raft_consensus.cc:2229] T 79d29cb7f2b64025be52c9774b821567 P a5f97462e246464da817a89feff3d0f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:04:07.061976 554 raft_consensus.cc:2258] T 79d29cb7f2b64025be52c9774b821567 P daee41db60d54eeaac1fe01e830d6d78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:04:07.061976 553 raft_consensus.cc:2258] T 79d29cb7f2b64025be52c9774b821567 P a5f97462e246464da817a89feff3d0f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:04:07.063769 554 ts_tablet_manager.cc:1905] T 79d29cb7f2b64025be52c9774b821567 P daee41db60d54eeaac1fe01e830d6d78: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:04:07.063769 553 ts_tablet_manager.cc:1905] T 79d29cb7f2b64025be52c9774b821567 P a5f97462e246464da817a89feff3d0f1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:04:07.066681 553 ts_tablet_manager.cc:1918] T 79d29cb7f2b64025be52c9774b821567 P a5f97462e246464da817a89feff3d0f1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:04:07.066681 554 ts_tablet_manager.cc:1918] T 79d29cb7f2b64025be52c9774b821567 P daee41db60d54eeaac1fe01e830d6d78: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:04:07.066798 553 log.cc:1196] T 79d29cb7f2b64025be52c9774b821567 P a5f97462e246464da817a89feff3d0f1: Deleting WAL directory at /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/minicluster-data/ts-2/wal/wals/79d29cb7f2b64025be52c9774b821567
I20260811 19:04:07.066798 554 log.cc:1196] T 79d29cb7f2b64025be52c9774b821567 P daee41db60d54eeaac1fe01e830d6d78: Deleting WAL directory at /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474792496492-13748-0/minicluster-data/ts-1/wal/wals/79d29cb7f2b64025be52c9774b821567
I20260811 19:04:07.067155 554 ts_tablet_manager.cc:1939] T 79d29cb7f2b64025be52c9774b821567 P daee41db60d54eeaac1fe01e830d6d78: Deleting consensus metadata
I20260811 19:04:07.067173 553 ts_tablet_manager.cc:1939] T 79d29cb7f2b64025be52c9774b821567 P a5f97462e246464da817a89feff3d0f1: Deleting consensus metadata
W20260811 19:04:07.068176 32438 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.109.1:40391 (error 108)
I20260811 19:04:07.068748 32438 catalog_manager.cc:4841] TS a5f97462e246464da817a89feff3d0f1 (127.13.109.3:37263): tablet 79d29cb7f2b64025be52c9774b821567 (table test-table [id=07c33e5257ab4852b0d1fe0bbd9cfc93]) successfully deleted
I20260811 19:04:07.068946 32438 catalog_manager.cc:4841] TS daee41db60d54eeaac1fe01e830d6d78 (127.13.109.2:39925): tablet 79d29cb7f2b64025be52c9774b821567 (table test-table [id=07c33e5257ab4852b0d1fe0bbd9cfc93]) successfully deleted
W20260811 19:04:07.069497 519 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:04:07.069210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.1:40391 (local address 127.0.0.1:39770)
0811 19:04:07.069314 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:07.069402 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.1:40391: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":28}
W20260811 19:04:07.069672 32438 catalog_manager.cc:4584] TS 37a05a83014340ce9366abd8cbfa972b (127.13.109.1:40391): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 79d29cb7f2b64025be52c9774b821567: Network error: Client connection negotiation failed: client connection to 127.13.109.1:40391: connect: Connection reset by peer (error 104)
I20260811 19:04:07.069741 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 32675
I20260811 19:04:07.078634 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 359
I20260811 19:04:07.087469 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 32419
2026-08-11T19:04:07Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2717 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-11T19:04:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:04:07Z Disabled control of system clock
I20260811 19:04:07.123550 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474792496492-13748-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474792496492-13748-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474792496492-13748-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.62:42631
--webserver_interface=127.13.109.62
--webserver_port=0
--builtin_ntp_servers=127.13.109.20:33383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.109.62:42631
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 19:04:07.243280 559 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:04:07.243597 559 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:04:07.243635 559 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:04:07.247785 559 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:04:07.247860 559 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:04:07.247877 559 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:04:07.247895 559 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:04:07.247911 559 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:04:07.253036 559 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:33383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474792496492-13748-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474792496492-13748-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.109.62:42631
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.62:42631
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.109.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.559
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474792496492-13748-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:04:07.254215 559 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:04:07.255597 559 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:04:07.264602 565 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:04:07.264734 559 server_base.cc:1034] running on GCE node
W20260811 19:04:07.264603 569 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:04:07.264626 567 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:04:07.265591 559 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:04:07.266809 559 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:04:07.268033 559 hybrid_clock.cc:648] HybridClock initialized: now 1786475047268013 us; error 51 us; skew 500 ppm
I20260811 19:04:07.270079 559 webserver.cc:460] Webserver started at http://127.13.109.62:41479/ using document root <none> and password file <none>
I20260811 19:04:07.270713 559 fs_manager.cc:359] Metadata directory not provided
I20260811 19:04:07.270824 559 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:04:07.271117 559 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:04:07.273401 559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474792496492-13748-0/minicluster-data/master-0/data/instance:
uuid: "9b724fe5c5e145fd8e5e3152bb5b1165"
format_stamp: "Formatted at 2026-08-11 19:04:07 on dist-test-slave-0kls"
server_key: "da0f7dd4e1041cc924afd14f06982fe1"
server_key_iv: "5c74ef992584a48d37e91cd615cd3d29"
server_key_version: "encryptionkey@0"
I20260811 19:04:07.274114 559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474792496492-13748-0/minicluster-data/master-0/wal/instance:
uuid: "9b724fe5c5e145fd8e5e3152bb5b1165"
format_stamp: "Formatted at 2026-08-11 19:04:07 on dist-test-slave-0kls"
server_key: "da0f7dd4e1041cc924afd14f06982fe1"
server_key_iv: "5c74ef992584a48d37e91cd615cd3d29"
server_key_version: "encryptionkey@0"
I20260811 19:04:07.278348 559 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 19:04:07.281162 576 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:04:07.282581 559 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 19:04:07.282704 559 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474792496492-13748-0/minicluster-data/master-0/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474792496492-13748-0/minicluster-data/master-0/wal
uuid: "9b724fe5c5e145fd8e5e3152bb5b1165"
format_stamp: "Formatted at 2026-08-11 19:04:07 on dist-test-slave-0kls"
server_key: "da0f7dd4e1041cc924afd14f06982fe1"
server_key_iv: "5c74ef992584a48d37e91cd615cd3d29"
server_key_version: "encryptionkey@0"
I20260811 19:04:07.282805 559 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474792496492-13748-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474792496492-13748-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474792496492-13748-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:04:07.308420 559 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:04:07.309190 559 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:04:07.309360 559 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:04:07.317350 559 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.62:42631
I20260811 19:04:07.317356 636 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.62:42631 every 8 connection(s)
I20260811 19:04:07.318427 559 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
I20260811 19:04:07.321357 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 559
I20260811 19:04:07.321465 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474792496492-13748-0/minicluster-data/master-0/wal/instance
I20260811 19:04:07.321863 637 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:04:07.321879 13748 external_mini_cluster.cc:1428] Setting key f02557fecb2e36e30e85fb652cb205cb
I20260811 19:04:07.328941 637 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b724fe5c5e145fd8e5e3152bb5b1165: Bootstrap starting.
I20260811 19:04:07.331481 637 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9b724fe5c5e145fd8e5e3152bb5b1165: Neither blocks nor log segments found. Creating new log.
I20260811 19:04:07.332079 640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:07.323558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49118 (local address 127.13.109.62:42631)
0811 19:04:07.324234 (+ 676us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:07.324251 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:07.324288 (+ 37us) server_negotiation.cc:407] Connection header received
0811 19:04:07.325066 (+ 778us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:07.325085 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:07.325489 (+ 404us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:07.325847 (+ 358us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:07.326219 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:07.327241 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:07.328491 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:07.328984 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:07.329217 (+ 233us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:07.330230 (+ 1013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:04:07.330235 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:04:07.330249 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:04:07.330655 (+ 406us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:04:07.330714 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:04:07.330735 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:04:07.330859 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:07.330979 (+ 120us) server_negotiation.cc:299] Negotiation successful
0811 19:04:07.331267 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":306,"thread_start_us":166,"threads_started":1}
I20260811 19:04:07.332347 637 log.cc:824] T 00000000000000000000000000000000 P 9b724fe5c5e145fd8e5e3152bb5b1165: Log is configured to *not* fsync() on all Append() calls
I20260811 19:04:07.334614 637 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b724fe5c5e145fd8e5e3152bb5b1165: No bootstrap required, opened a new log
I20260811 19:04:07.338287 637 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9b724fe5c5e145fd8e5e3152bb5b1165 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b724fe5c5e145fd8e5e3152bb5b1165" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 42631 } }
I20260811 19:04:07.338531 637 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9b724fe5c5e145fd8e5e3152bb5b1165 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:04:07.338575 637 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9b724fe5c5e145fd8e5e3152bb5b1165 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b724fe5c5e145fd8e5e3152bb5b1165, State: Initialized, Role: FOLLOWER
I20260811 19:04:07.339010 637 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9b724fe5c5e145fd8e5e3152bb5b1165 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b724fe5c5e145fd8e5e3152bb5b1165" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 42631 } }
I20260811 19:04:07.339200 637 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9b724fe5c5e145fd8e5e3152bb5b1165 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:04:07.339267 637 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9b724fe5c5e145fd8e5e3152bb5b1165 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:04:07.339375 637 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9b724fe5c5e145fd8e5e3152bb5b1165 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:04:07.340408 637 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9b724fe5c5e145fd8e5e3152bb5b1165 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b724fe5c5e145fd8e5e3152bb5b1165" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 42631 } }
I20260811 19:04:07.340785 637 leader_election.cc:302] T 00000000000000000000000000000000 P 9b724fe5c5e145fd8e5e3152bb5b1165 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9b724fe5c5e145fd8e5e3152bb5b1165; no voters:
I20260811 19:04:07.341140 637 leader_election.cc:288] T 00000000000000000000000000000000 P 9b724fe5c5e145fd8e5e3152bb5b1165 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:04:07.341241 643 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9b724fe5c5e145fd8e5e3152bb5b1165 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:04:07.341585 643 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9b724fe5c5e145fd8e5e3152bb5b1165 [term 1 LEADER]: Becoming Leader. State: Replica: 9b724fe5c5e145fd8e5e3152bb5b1165, State: Running, Role: LEADER
I20260811 19:04:07.341984 643 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9b724fe5c5e145fd8e5e3152bb5b1165 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b724fe5c5e145fd8e5e3152bb5b1165" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 42631 } }
I20260811 19:04:07.342576 637 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9b724fe5c5e145fd8e5e3152bb5b1165 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:04:07.343840 646 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b724fe5c5e145fd8e5e3152bb5b1165 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9b724fe5c5e145fd8e5e3152bb5b1165. Latest consensus state: current_term: 1 leader_uuid: "9b724fe5c5e145fd8e5e3152bb5b1165" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b724fe5c5e145fd8e5e3152bb5b1165" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 42631 } } }
I20260811 19:04:07.344005 646 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b724fe5c5e145fd8e5e3152bb5b1165 [sys.catalog]: This master's current role is: LEADER
I20260811 19:04:07.343845 644 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b724fe5c5e145fd8e5e3152bb5b1165 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9b724fe5c5e145fd8e5e3152bb5b1165" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b724fe5c5e145fd8e5e3152bb5b1165" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 42631 } } }
I20260811 19:04:07.344362 644 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b724fe5c5e145fd8e5e3152bb5b1165 [sys.catalog]: This master's current role is: LEADER
I20260811 19:04:07.344445 654 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:04:07.347777 654 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:04:07.354496 654 catalog_manager.cc:1347] Generated new cluster ID: a66fcc3fa22e425eb43d614ba2b802a8
I20260811 19:04:07.354594 654 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:04:07.373302 654 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:04:07.374697 654 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:04:07.392695 654 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9b724fe5c5e145fd8e5e3152bb5b1165: Generated new TSK 0
I20260811 19:04:07.393734 654 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 19:04:07.397993 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474792496492-13748-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474792496492-13748-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474792496492-13748-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.1:0
--local_ip_for_outbound_sockets=127.13.109.1
--webserver_interface=127.13.109.1
--webserver_port=0
--tserver_master_addrs=127.13.109.62:42631
--builtin_ntp_servers=127.13.109.20:33383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 19:04:07.520388 665 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:04:07.520744 665 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:04:07.520828 665 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:04:07.525074 665 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:04:07.525213 665 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:04:07.525316 665 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.1
I20260811 19:04:07.530622 665 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:33383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474792496492-13748-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474792496492-13748-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.109.1
--webserver_port=0
--tserver_master_addrs=127.13.109.62:42631
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.665
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.1
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474792496492-13748-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:04:07.531924 665 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:04:07.533432 665 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 19:04:07.544222 665 server_base.cc:1034] running on GCE node
W20260811 19:04:07.544106 673 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:04:07.544185 670 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:04:07.544162 671 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:04:07.544960 665 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:04:07.545749 665 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:04:07.546975 665 hybrid_clock.cc:648] HybridClock initialized: now 1786475047546961 us; error 29 us; skew 500 ppm
I20260811 19:04:07.548971 665 webserver.cc:460] Webserver started at http://127.13.109.1:42615/ using document root <none> and password file <none>
I20260811 19:04:07.549562 665 fs_manager.cc:359] Metadata directory not provided
I20260811 19:04:07.549626 665 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:04:07.549892 665 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:04:07.552080 665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474792496492-13748-0/minicluster-data/ts-0/data/instance:
uuid: "e852feeaf434478d93a92447d91c89b9"
format_stamp: "Formatted at 2026-08-11 19:04:07 on dist-test-slave-0kls"
server_key: "649d7e49d96f8715af583ab13707dc9e"
server_key_iv: "b26f8ce1a0dbb8e93ae491cef2f09b1c"
server_key_version: "encryptionkey@0"
I20260811 19:04:07.552626 665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474792496492-13748-0/minicluster-data/ts-0/wal/instance:
uuid: "e852feeaf434478d93a92447d91c89b9"
format_stamp: "Formatted at 2026-08-11 19:04:07 on dist-test-slave-0kls"
server_key: "649d7e49d96f8715af583ab13707dc9e"
server_key_iv: "b26f8ce1a0dbb8e93ae491cef2f09b1c"
server_key_version: "encryptionkey@0"
I20260811 19:04:07.556684 665 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.005s
I20260811 19:04:07.559326 679 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:04:07.560688 665 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 19:04:07.560812 665 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474792496492-13748-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474792496492-13748-0/minicluster-data/ts-0/wal
uuid: "e852feeaf434478d93a92447d91c89b9"
format_stamp: "Formatted at 2026-08-11 19:04:07 on dist-test-slave-0kls"
server_key: "649d7e49d96f8715af583ab13707dc9e"
server_key_iv: "b26f8ce1a0dbb8e93ae491cef2f09b1c"
server_key_version: "encryptionkey@0"
I20260811 19:04:07.560935 665 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474792496492-13748-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474792496492-13748-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474792496492-13748-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:04:07.580530 665 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:04:07.581317 665 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:04:07.581538 665 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:04:07.582305 665 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:04:07.583364 665 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:04:07.583412 665 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:04:07.583454 665 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:04:07.583514 665 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:04:07.594200 665 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.1:37477
I20260811 19:04:07.594237 810 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.1:37477 every 8 connection(s)
I20260811 19:04:07.595196 665 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
I20260811 19:04:07.596724 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 665
I20260811 19:04:07.596849 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474792496492-13748-0/minicluster-data/ts-0/wal/instance
I20260811 19:04:07.597162 13748 external_mini_cluster.cc:1428] Setting key 4eb75463f345ad3f8572109b1d2df6b4
I20260811 19:04:07.599470 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474792496492-13748-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474792496492-13748-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474792496492-13748-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.2:0
--local_ip_for_outbound_sockets=127.13.109.2
--webserver_interface=127.13.109.2
--webserver_port=0
--tserver_master_addrs=127.13.109.62:42631
--builtin_ntp_servers=127.13.109.20:33383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 19:04:07.607048 640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:07.597382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.1:49691 (local address 127.13.109.62:42631)
0811 19:04:07.597522 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:07.597526 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:07.599005 (+ 1479us) server_negotiation.cc:407] Connection header received
0811 19:04:07.601087 (+ 2082us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:07.601090 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:07.601135 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:07.601209 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:07.602326 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:07.602863 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:07.604675 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:07.604899 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:07.605059 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:07.606523 (+ 1464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:04:07.606526 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:04:07.606528 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:04:07.606541 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:04:07.606561 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:04:07.606569 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:04:07.606629 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:07.606860 (+ 231us) server_negotiation.cc:299] Negotiation successful
0811 19:04:07.606923 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 19:04:07.608408 813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:07.597969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:42631 (local address 127.13.109.1:49691)
0811 19:04:07.598825 (+ 856us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:07.598881 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 19:04:07.600839 (+ 1958us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:04:07.601456 (+ 617us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:04:07.601469 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:04:07.601598 (+ 129us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:04:07.602170 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:07.602184 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:07.603018 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:07.603023 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:07.604516 (+ 1493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:07.604541 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:07.605136 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:07.605142 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:07.605398 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:07.606168 (+ 770us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 19:04:07.606172 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:04:07.606236 (+ 64us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 19:04:07.606338 (+ 102us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:04:07.606667 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:04:07.606680 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:04:07.606689 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 19:04:07.606836 (+ 147us) client_negotiation.cc:240] Negotiation successful
0811 19:04:07.607191 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":164,"threads_started":1}
I20260811 19:04:07.609855 811 heartbeater.cc:347] Connected to a master server at 127.13.109.62:42631
I20260811 19:04:07.610145 811 heartbeater.cc:464] Registering TS with master...
I20260811 19:04:07.610896 811 heartbeater.cc:511] Master 127.13.109.62:42631 requested a full tablet report, sending...
I20260811 19:04:07.612641 597 ts_manager.cc:194] Registered new tserver with Master: e852feeaf434478d93a92447d91c89b9 (127.13.109.1:37477)
I20260811 19:04:07.613945 597 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.109.1:49691
W20260811 19:04:07.722220 814 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:04:07.722577 814 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:04:07.722620 814 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:04:07.730322 814 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:04:07.730401 814 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:04:07.730511 814 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.2
I20260811 19:04:07.738785 814 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:33383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474792496492-13748-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474792496492-13748-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.109.2
--webserver_port=0
--tserver_master_addrs=127.13.109.62:42631
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.814
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.2
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474792496492-13748-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:04:07.740624 814 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:04:07.742806 814 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:04:07.754295 822 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:04:07.754298 820 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:04:07.754575 814 server_base.cc:1034] running on GCE node
W20260811 19:04:07.754298 819 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:04:07.755242 814 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:04:07.756016 814 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:04:07.757210 814 hybrid_clock.cc:648] HybridClock initialized: now 1786475047757194 us; error 30 us; skew 500 ppm
I20260811 19:04:07.759444 814 webserver.cc:460] Webserver started at http://127.13.109.2:33381/ using document root <none> and password file <none>
I20260811 19:04:07.760136 814 fs_manager.cc:359] Metadata directory not provided
I20260811 19:04:07.760210 814 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:04:07.760499 814 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:04:07.762915 814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474792496492-13748-0/minicluster-data/ts-1/data/instance:
uuid: "b9a3bccd535d4423aadedeed463a3088"
format_stamp: "Formatted at 2026-08-11 19:04:07 on dist-test-slave-0kls"
server_key: "0c9891512df9558c2ae34d2160390f06"
server_key_iv: "2968bad533ec07b1aa46a0107b29a1a0"
server_key_version: "encryptionkey@0"
I20260811 19:04:07.763574 814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474792496492-13748-0/minicluster-data/ts-1/wal/instance:
uuid: "b9a3bccd535d4423aadedeed463a3088"
format_stamp: "Formatted at 2026-08-11 19:04:07 on dist-test-slave-0kls"
server_key: "0c9891512df9558c2ae34d2160390f06"
server_key_iv: "2968bad533ec07b1aa46a0107b29a1a0"
server_key_version: "encryptionkey@0"
I20260811 19:04:07.768054 814 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 19:04:07.770928 828 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:04:07.772346 814 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 19:04:07.772498 814 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474792496492-13748-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474792496492-13748-0/minicluster-data/ts-1/wal
uuid: "b9a3bccd535d4423aadedeed463a3088"
format_stamp: "Formatted at 2026-08-11 19:04:07 on dist-test-slave-0kls"
server_key: "0c9891512df9558c2ae34d2160390f06"
server_key_iv: "2968bad533ec07b1aa46a0107b29a1a0"
server_key_version: "encryptionkey@0"
I20260811 19:04:07.772637 814 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474792496492-13748-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474792496492-13748-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474792496492-13748-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:04:07.795488 814 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:04:07.796303 814 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:04:07.796590 814 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:04:07.797284 814 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:04:07.798381 814 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:04:07.798430 814 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:04:07.798511 814 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:04:07.798553 814 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:04:07.808019 814 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.2:35775
I20260811 19:04:07.808048 955 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.2:35775 every 8 connection(s)
I20260811 19:04:07.809046 814 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
I20260811 19:04:07.819025 640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:07.811155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.2:57685 (local address 127.13.109.62:42631)
0811 19:04:07.811307 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:07.811311 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:07.812502 (+ 1191us) server_negotiation.cc:407] Connection header received
0811 19:04:07.814026 (+ 1524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:07.814030 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:07.814095 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:07.814209 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:07.815175 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:07.815717 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:07.816935 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:07.817183 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:07.817312 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:07.818469 (+ 1157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:04:07.818477 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:04:07.818479 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:04:07.818493 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:04:07.818515 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:04:07.818522 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:04:07.818615 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:07.818788 (+ 173us) server_negotiation.cc:299] Negotiation successful
0811 19:04:07.818865 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 19:04:07.819274 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 814
I20260811 19:04:07.819413 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474792496492-13748-0/minicluster-data/ts-1/wal/instance
I20260811 19:04:07.819779 13748 external_mini_cluster.cc:1428] Setting key 26b2bb7b07d37fa600c9670b4a13252c
I20260811 19:04:07.820185 958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:07.811530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:42631 (local address 127.13.109.2:57685)
0811 19:04:07.812332 (+ 802us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:07.812376 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 19:04:07.813761 (+ 1385us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:04:07.814397 (+ 636us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:04:07.814404 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:04:07.814491 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:04:07.814998 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:07.815011 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:07.815864 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:07.815868 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:07.816787 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:07.816798 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:07.817363 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:07.817368 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:07.817547 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:07.818179 (+ 632us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 19:04:07.818180 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:04:07.818224 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 19:04:07.818296 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:04:07.818628 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:04:07.818640 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:04:07.818652 (+ 12us) client_negotiation.cc:769] Sending connection context
0811 19:04:07.818788 (+ 136us) client_negotiation.cc:240] Negotiation successful
0811 19:04:07.819152 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":160,"threads_started":1}
I20260811 19:04:07.821579 956 heartbeater.cc:347] Connected to a master server at 127.13.109.62:42631
I20260811 19:04:07.821902 956 heartbeater.cc:464] Registering TS with master...
I20260811 19:04:07.822222 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474792496492-13748-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474792496492-13748-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474792496492-13748-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.3:0
--local_ip_for_outbound_sockets=127.13.109.3
--webserver_interface=127.13.109.3
--webserver_port=0
--tserver_master_addrs=127.13.109.62:42631
--builtin_ntp_servers=127.13.109.20:33383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 19:04:07.822570 956 heartbeater.cc:511] Master 127.13.109.62:42631 requested a full tablet report, sending...
I20260811 19:04:07.823770 597 ts_manager.cc:194] Registered new tserver with Master: b9a3bccd535d4423aadedeed463a3088 (127.13.109.2:35775)
I20260811 19:04:07.824654 597 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.109.2:57685
W20260811 19:04:07.942087 959 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:04:07.942444 959 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:04:07.942485 959 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:04:07.946882 959 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:04:07.946956 959 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:04:07.947016 959 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.3
I20260811 19:04:07.952062 959 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:33383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474792496492-13748-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474792496492-13748-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.109.3
--webserver_port=0
--tserver_master_addrs=127.13.109.62:42631
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.959
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.3
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474792496492-13748-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:04:07.953181 959 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:04:07.954586 959 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:04:07.964818 964 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:04:07.964843 959 server_base.cc:1034] running on GCE node
W20260811 19:04:07.964826 965 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:04:07.964855 967 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:04:07.965595 959 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:04:07.966348 959 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:04:07.967557 959 hybrid_clock.cc:648] HybridClock initialized: now 1786475047967522 us; error 50 us; skew 500 ppm
I20260811 19:04:07.969497 959 webserver.cc:460] Webserver started at http://127.13.109.3:38135/ using document root <none> and password file <none>
I20260811 19:04:07.970204 959 fs_manager.cc:359] Metadata directory not provided
I20260811 19:04:07.970307 959 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:04:07.970566 959 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:04:07.972677 959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474792496492-13748-0/minicluster-data/ts-2/data/instance:
uuid: "04b7bcf259a842999c22a3c0c4e4c11f"
format_stamp: "Formatted at 2026-08-11 19:04:07 on dist-test-slave-0kls"
server_key: "051ad3b4c0a43714cb222773447f0175"
server_key_iv: "bbc6d50340be06f0e58d198f15bf9db1"
server_key_version: "encryptionkey@0"
I20260811 19:04:07.973526 959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474792496492-13748-0/minicluster-data/ts-2/wal/instance:
uuid: "04b7bcf259a842999c22a3c0c4e4c11f"
format_stamp: "Formatted at 2026-08-11 19:04:07 on dist-test-slave-0kls"
server_key: "051ad3b4c0a43714cb222773447f0175"
server_key_iv: "bbc6d50340be06f0e58d198f15bf9db1"
server_key_version: "encryptionkey@0"
I20260811 19:04:07.977628 959 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260811 19:04:07.980173 974 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:04:07.981499 959 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 19:04:07.981668 959 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474792496492-13748-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474792496492-13748-0/minicluster-data/ts-2/wal
uuid: "04b7bcf259a842999c22a3c0c4e4c11f"
format_stamp: "Formatted at 2026-08-11 19:04:07 on dist-test-slave-0kls"
server_key: "051ad3b4c0a43714cb222773447f0175"
server_key_iv: "bbc6d50340be06f0e58d198f15bf9db1"
server_key_version: "encryptionkey@0"
I20260811 19:04:07.981794 959 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474792496492-13748-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474792496492-13748-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474792496492-13748-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:04:08.001389 959 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:04:08.002568 959 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:04:08.002840 959 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:04:08.003762 959 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:04:08.005322 959 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:04:08.005411 959 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:04:08.005471 959 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:04:08.005555 959 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260811 19:04:08.015813 959 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.3:34755
I20260811 19:04:08.015832 1103 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.3:34755 every 8 connection(s)
I20260811 19:04:08.016778 959 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
I20260811 19:04:08.021531 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 959
I20260811 19:04:08.021646 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474792496492-13748-0/minicluster-data/ts-2/wal/instance
I20260811 19:04:08.022010 13748 external_mini_cluster.cc:1428] Setting key 2f30f99eea8e1d3ee1080d596e552b5f
I20260811 19:04:08.026732 640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:08.018706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.3:48463 (local address 127.13.109.62:42631)
0811 19:04:08.018850 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:08.018854 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:08.019863 (+ 1009us) server_negotiation.cc:407] Connection header received
0811 19:04:08.021301 (+ 1438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:08.021308 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:08.021376 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:08.021482 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:08.022595 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:08.023361 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:08.024628 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:08.024877 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:08.025001 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:08.026175 (+ 1174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:04:08.026182 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:04:08.026185 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:04:08.026199 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:04:08.026222 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:04:08.026232 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:04:08.026322 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:08.026495 (+ 173us) server_negotiation.cc:299] Negotiation successful
0811 19:04:08.026572 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 19:04:08.027690 1106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:08.019044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:42631 (local address 127.13.109.3:48463)
0811 19:04:08.019704 (+ 660us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:08.019767 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0811 19:04:08.021028 (+ 1261us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:04:08.021683 (+ 655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:04:08.021694 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:04:08.021841 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:04:08.022436 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:08.022451 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:08.023503 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:08.023511 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:08.024498 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:08.024508 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:08.025047 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:08.025051 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:08.025229 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:08.025894 (+ 665us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 19:04:08.025895 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:04:08.025939 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 19:04:08.026012 (+ 73us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:04:08.026336 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:04:08.026348 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:04:08.026359 (+ 11us) client_negotiation.cc:769] Sending connection context
0811 19:04:08.026496 (+ 137us) client_negotiation.cc:240] Negotiation successful
0811 19:04:08.026807 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":121,"threads_started":1}
I20260811 19:04:08.028863 1104 heartbeater.cc:347] Connected to a master server at 127.13.109.62:42631
I20260811 19:04:08.029126 1104 heartbeater.cc:464] Registering TS with master...
I20260811 19:04:08.029713 1104 heartbeater.cc:511] Master 127.13.109.62:42631 requested a full tablet report, sending...
I20260811 19:04:08.030910 597 ts_manager.cc:194] Registered new tserver with Master: 04b7bcf259a842999c22a3c0c4e4c11f (127.13.109.3:34755)
I20260811 19:04:08.031579 597 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.109.3:48463
I20260811 19:04:08.036186 13748 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 19:04:08.042344 640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:08.038331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49122 (local address 127.13.109.62:42631)
0811 19:04:08.038469 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:08.038474 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:08.038637 (+ 163us) server_negotiation.cc:407] Connection header received
0811 19:04:08.038818 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:08.038826 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:08.038870 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:08.038962 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:08.039251 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:08.039786 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:08.040651 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:08.040878 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:08.041000 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:08.041829 (+ 829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:04:08.041835 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:04:08.041838 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:04:08.041852 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:04:08.041875 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:04:08.041892 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:04:08.041978 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:08.042110 (+ 132us) server_negotiation.cc:299] Negotiation successful
0811 19:04:08.042175 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 19:04:08.045300 597 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49122:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 19:04:08.048470 597 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 19:04:08.064458 1118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:08.058145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43824 (local address 127.13.109.1:37477)
0811 19:04:08.059075 (+ 930us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:08.059089 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:08.059118 (+ 29us) server_negotiation.cc:407] Connection header received
0811 19:04:08.059213 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:08.059219 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:08.059435 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:08.059602 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:04:08.059999 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:08.061300 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:08.063047 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:08.063636 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:08.063794 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:08.064111 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:08.064178 (+ 67us) server_negotiation.cc:299] Negotiation successful
0811 19:04:08.064295 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":813,"thread_start_us":175,"threads_started":1}
I20260811 19:04:08.064438 1117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:08.057681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.1:37477 (local address 127.0.0.1:43824)
0811 19:04:08.058561 (+ 880us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:08.058621 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 19:04:08.058772 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:04:08.059617 (+ 845us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:04:08.059624 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:04:08.059657 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:04:08.059844 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:08.059852 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:08.061441 (+ 1589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:08.061444 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:08.062903 (+ 1459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:08.062913 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:08.063796 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:08.063800 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:08.063918 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:08.063961 (+ 43us) client_negotiation.cc:769] Sending connection context
0811 19:04:08.064079 (+ 118us) client_negotiation.cc:240] Negotiation successful
0811 19:04:08.064235 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":700,"mutex_wait_us":62,"spinlock_wait_cycles":7424,"thread_start_us":129,"threads_started":1}
I20260811 19:04:08.064437 1116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:08.057609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.3:34755 (local address 127.0.0.1:40980)
0811 19:04:08.058565 (+ 956us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:08.058619 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 19:04:08.058766 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:04:08.060052 (+ 1286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:04:08.060055 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:04:08.060076 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:04:08.060183 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:08.060188 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:08.061304 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:08.061310 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:08.062864 (+ 1554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:08.062878 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:08.064002 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:08.064006 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:08.064093 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:08.064120 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 19:04:08.064171 (+ 51us) client_negotiation.cc:240] Negotiation successful
0811 19:04:08.064251 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":226,"mutex_wait_us":118,"spinlock_wait_cycles":1455232,"thread_start_us":126,"threads_started":1}
I20260811 19:04:08.065550 1119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:08.058161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40980 (local address 127.13.109.3:34755)
0811 19:04:08.059549 (+ 1388us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:08.059564 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:08.059591 (+ 27us) server_negotiation.cc:407] Connection header received
0811 19:04:08.059666 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:08.059671 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:08.059896 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:08.060315 (+ 419us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:04:08.060434 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:08.061159 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:08.063018 (+ 1859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:08.063861 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:08.064886 (+ 1025us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:08.065180 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:08.065243 (+ 63us) server_negotiation.cc:299] Negotiation successful
0811 19:04:08.065359 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1249,"thread_start_us":157,"threads_started":1}
I20260811 19:04:08.066007 1120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:08.058466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.2:35775 (local address 127.0.0.1:38782)
0811 19:04:08.058995 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:08.059007 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 19:04:08.059084 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:04:08.059811 (+ 727us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:04:08.059814 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:04:08.059831 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:04:08.059951 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:08.059958 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:08.061482 (+ 1524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:08.061485 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:08.063812 (+ 2327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:08.063826 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:08.065513 (+ 1687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:08.065517 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:08.065631 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:08.065663 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 19:04:08.065740 (+ 77us) client_negotiation.cc:240] Negotiation successful
0811 19:04:08.065861 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"mutex_wait_us":1181,"thread_start_us":87,"threads_started":1}
I20260811 19:04:08.066648 1121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:08.058724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38782 (local address 127.13.109.2:35775)
0811 19:04:08.059151 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:08.059162 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:08.059180 (+ 18us) server_negotiation.cc:407] Connection header received
0811 19:04:08.059245 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:08.059249 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:08.059435 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:08.059601 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:04:08.060095 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:08.061301 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:08.063991 (+ 2690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:08.065140 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:08.065268 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:08.065440 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:08.066363 (+ 923us) server_negotiation.cc:299] Negotiation successful
0811 19:04:08.066477 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":127,"threads_started":1}
I20260811 19:04:08.068331 1028 tablet_service.cc:1467] Processing CreateTablet for tablet 0f3c8994480948e9a7e74e1616d24991 (DEFAULT_TABLE table=test-table [id=1d3e7f5edc1d4c63a5fee272d1de2e55]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:04:08.068362 735 tablet_service.cc:1467] Processing CreateTablet for tablet 0f3c8994480948e9a7e74e1616d24991 (DEFAULT_TABLE table=test-table [id=1d3e7f5edc1d4c63a5fee272d1de2e55]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:04:08.069478 735 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f3c8994480948e9a7e74e1616d24991. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:04:08.069479 1028 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f3c8994480948e9a7e74e1616d24991. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:04:08.069371 883 tablet_service.cc:1467] Processing CreateTablet for tablet 0f3c8994480948e9a7e74e1616d24991 (DEFAULT_TABLE table=test-table [id=1d3e7f5edc1d4c63a5fee272d1de2e55]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:04:08.070474 883 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f3c8994480948e9a7e74e1616d24991. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:04:08.076380 1122 tablet_bootstrap.cc:492] T 0f3c8994480948e9a7e74e1616d24991 P 04b7bcf259a842999c22a3c0c4e4c11f: Bootstrap starting.
I20260811 19:04:08.077920 1123 tablet_bootstrap.cc:492] T 0f3c8994480948e9a7e74e1616d24991 P e852feeaf434478d93a92447d91c89b9: Bootstrap starting.
I20260811 19:04:08.077916 1124 tablet_bootstrap.cc:492] T 0f3c8994480948e9a7e74e1616d24991 P b9a3bccd535d4423aadedeed463a3088: Bootstrap starting.
I20260811 19:04:08.079701 1122 tablet_bootstrap.cc:654] T 0f3c8994480948e9a7e74e1616d24991 P 04b7bcf259a842999c22a3c0c4e4c11f: Neither blocks nor log segments found. Creating new log.
I20260811 19:04:08.079770 1123 tablet_bootstrap.cc:654] T 0f3c8994480948e9a7e74e1616d24991 P e852feeaf434478d93a92447d91c89b9: Neither blocks nor log segments found. Creating new log.
I20260811 19:04:08.080032 1124 tablet_bootstrap.cc:654] T 0f3c8994480948e9a7e74e1616d24991 P b9a3bccd535d4423aadedeed463a3088: Neither blocks nor log segments found. Creating new log.
I20260811 19:04:08.080729 1123 log.cc:824] T 0f3c8994480948e9a7e74e1616d24991 P e852feeaf434478d93a92447d91c89b9: Log is configured to *not* fsync() on all Append() calls
I20260811 19:04:08.080847 1122 log.cc:824] T 0f3c8994480948e9a7e74e1616d24991 P 04b7bcf259a842999c22a3c0c4e4c11f: Log is configured to *not* fsync() on all Append() calls
I20260811 19:04:08.080847 1124 log.cc:824] T 0f3c8994480948e9a7e74e1616d24991 P b9a3bccd535d4423aadedeed463a3088: Log is configured to *not* fsync() on all Append() calls
I20260811 19:04:08.082926 1122 tablet_bootstrap.cc:492] T 0f3c8994480948e9a7e74e1616d24991 P 04b7bcf259a842999c22a3c0c4e4c11f: No bootstrap required, opened a new log
I20260811 19:04:08.083022 1123 tablet_bootstrap.cc:492] T 0f3c8994480948e9a7e74e1616d24991 P e852feeaf434478d93a92447d91c89b9: No bootstrap required, opened a new log
I20260811 19:04:08.083022 1124 tablet_bootstrap.cc:492] T 0f3c8994480948e9a7e74e1616d24991 P b9a3bccd535d4423aadedeed463a3088: No bootstrap required, opened a new log
I20260811 19:04:08.083195 1122 ts_tablet_manager.cc:1397] T 0f3c8994480948e9a7e74e1616d24991 P 04b7bcf259a842999c22a3c0c4e4c11f: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260811 19:04:08.083214 1123 ts_tablet_manager.cc:1397] T 0f3c8994480948e9a7e74e1616d24991 P e852feeaf434478d93a92447d91c89b9: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.001s
I20260811 19:04:08.083214 1124 ts_tablet_manager.cc:1397] T 0f3c8994480948e9a7e74e1616d24991 P b9a3bccd535d4423aadedeed463a3088: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.001s
I20260811 19:04:08.087494 1122 raft_consensus.cc:348] T 0f3c8994480948e9a7e74e1616d24991 P 04b7bcf259a842999c22a3c0c4e4c11f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04b7bcf259a842999c22a3c0c4e4c11f" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 34755 } } peers { permanent_uuid: "e852feeaf434478d93a92447d91c89b9" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 37477 } } peers { permanent_uuid: "b9a3bccd535d4423aadedeed463a3088" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 35775 } }
I20260811 19:04:08.087695 1123 raft_consensus.cc:348] T 0f3c8994480948e9a7e74e1616d24991 P e852feeaf434478d93a92447d91c89b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04b7bcf259a842999c22a3c0c4e4c11f" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 34755 } } peers { permanent_uuid: "e852feeaf434478d93a92447d91c89b9" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 37477 } } peers { permanent_uuid: "b9a3bccd535d4423aadedeed463a3088" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 35775 } }
I20260811 19:04:08.087697 1124 raft_consensus.cc:348] T 0f3c8994480948e9a7e74e1616d24991 P b9a3bccd535d4423aadedeed463a3088 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04b7bcf259a842999c22a3c0c4e4c11f" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 34755 } } peers { permanent_uuid: "e852feeaf434478d93a92447d91c89b9" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 37477 } } peers { permanent_uuid: "b9a3bccd535d4423aadedeed463a3088" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 35775 } }
I20260811 19:04:08.087920 1122 raft_consensus.cc:374] T 0f3c8994480948e9a7e74e1616d24991 P 04b7bcf259a842999c22a3c0c4e4c11f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:04:08.087975 1124 raft_consensus.cc:374] T 0f3c8994480948e9a7e74e1616d24991 P b9a3bccd535d4423aadedeed463a3088 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:04:08.087983 1122 raft_consensus.cc:729] T 0f3c8994480948e9a7e74e1616d24991 P 04b7bcf259a842999c22a3c0c4e4c11f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04b7bcf259a842999c22a3c0c4e4c11f, State: Initialized, Role: FOLLOWER
I20260811 19:04:08.088011 1123 raft_consensus.cc:374] T 0f3c8994480948e9a7e74e1616d24991 P e852feeaf434478d93a92447d91c89b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:04:08.088063 1124 raft_consensus.cc:729] T 0f3c8994480948e9a7e74e1616d24991 P b9a3bccd535d4423aadedeed463a3088 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9a3bccd535d4423aadedeed463a3088, State: Initialized, Role: FOLLOWER
I20260811 19:04:08.088086 1123 raft_consensus.cc:729] T 0f3c8994480948e9a7e74e1616d24991 P e852feeaf434478d93a92447d91c89b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e852feeaf434478d93a92447d91c89b9, State: Initialized, Role: FOLLOWER
I20260811 19:04:08.088497 1122 consensus_queue.cc:260] T 0f3c8994480948e9a7e74e1616d24991 P 04b7bcf259a842999c22a3c0c4e4c11f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04b7bcf259a842999c22a3c0c4e4c11f" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 34755 } } peers { permanent_uuid: "e852feeaf434478d93a92447d91c89b9" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 37477 } } peers { permanent_uuid: "b9a3bccd535d4423aadedeed463a3088" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 35775 } }
I20260811 19:04:08.088580 1123 consensus_queue.cc:260] T 0f3c8994480948e9a7e74e1616d24991 P e852feeaf434478d93a92447d91c89b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04b7bcf259a842999c22a3c0c4e4c11f" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 34755 } } peers { permanent_uuid: "e852feeaf434478d93a92447d91c89b9" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 37477 } } peers { permanent_uuid: "b9a3bccd535d4423aadedeed463a3088" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 35775 } }
I20260811 19:04:08.088580 1124 consensus_queue.cc:260] T 0f3c8994480948e9a7e74e1616d24991 P b9a3bccd535d4423aadedeed463a3088 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04b7bcf259a842999c22a3c0c4e4c11f" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 34755 } } peers { permanent_uuid: "e852feeaf434478d93a92447d91c89b9" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 37477 } } peers { permanent_uuid: "b9a3bccd535d4423aadedeed463a3088" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 35775 } }
I20260811 19:04:08.089608 1122 ts_tablet_manager.cc:1428] T 0f3c8994480948e9a7e74e1616d24991 P 04b7bcf259a842999c22a3c0c4e4c11f: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 19:04:08.089679 1123 ts_tablet_manager.cc:1428] T 0f3c8994480948e9a7e74e1616d24991 P e852feeaf434478d93a92447d91c89b9: Time spent starting tablet: real 0.006s user 0.004s sys 0.002s
I20260811 19:04:08.089680 1124 ts_tablet_manager.cc:1428] T 0f3c8994480948e9a7e74e1616d24991 P b9a3bccd535d4423aadedeed463a3088: Time spent starting tablet: real 0.006s user 0.004s sys 0.002s
I20260811 19:04:08.090062 956 heartbeater.cc:503] Master 127.13.109.62:42631 was elected leader, sending a full tablet report...
I20260811 19:04:08.090068 1104 heartbeater.cc:503] Master 127.13.109.62:42631 was elected leader, sending a full tablet report...
I20260811 19:04:08.091136 811 heartbeater.cc:503] Master 127.13.109.62:42631 was elected leader, sending a full tablet report...
W20260811 19:04:08.096410 812 tablet.cc:2369] T 0f3c8994480948e9a7e74e1616d24991 P e852feeaf434478d93a92447d91c89b9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 19:04:08.173007 1130 raft_consensus.cc:482] T 0f3c8994480948e9a7e74e1616d24991 P e852feeaf434478d93a92447d91c89b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:04:08.173298 1130 raft_consensus.cc:504] T 0f3c8994480948e9a7e74e1616d24991 P e852feeaf434478d93a92447d91c89b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04b7bcf259a842999c22a3c0c4e4c11f" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 34755 } } peers { permanent_uuid: "e852feeaf434478d93a92447d91c89b9" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 37477 } } peers { permanent_uuid: "b9a3bccd535d4423aadedeed463a3088" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 35775 } }
I20260811 19:04:08.174858 1130 leader_election.cc:288] T 0f3c8994480948e9a7e74e1616d24991 P e852feeaf434478d93a92447d91c89b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 04b7bcf259a842999c22a3c0c4e4c11f (127.13.109.3:34755), b9a3bccd535d4423aadedeed463a3088 (127.13.109.2:35775)
I20260811 19:04:08.179417 1132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:08.175301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.3:34755 (local address 127.13.109.1:52241)
0811 19:04:08.175616 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:08.175635 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 19:04:08.175776 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:04:08.176203 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:04:08.176206 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:04:08.176226 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:04:08.176418 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:08.176426 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:08.177253 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:08.177258 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:08.178217 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:08.178227 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:08.178927 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:08.178931 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:08.179037 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:08.179067 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 19:04:08.179167 (+ 100us) client_negotiation.cc:240] Negotiation successful
0811 19:04:08.179246 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"mutex_wait_us":3,"thread_start_us":116,"threads_started":1}
I20260811 19:04:08.179399 1119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:08.175418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.1:52241 (local address 127.13.109.3:34755)
0811 19:04:08.175601 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:08.175605 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:08.175727 (+ 122us) server_negotiation.cc:407] Connection header received
0811 19:04:08.175994 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:08.175999 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:08.176061 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:08.176173 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:04:08.176574 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:08.177122 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:08.178361 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:08.178784 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:08.178932 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:08.178989 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:08.179187 (+ 198us) server_negotiation.cc:299] Negotiation successful
0811 19:04:08.179266 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260811 19:04:08.179832 1121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:08.175906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.1:57619 (local address 127.13.109.2:35775)
0811 19:04:08.176064 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:08.176070 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:08.176277 (+ 207us) server_negotiation.cc:407] Connection header received
0811 19:04:08.176446 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:08.176449 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:08.176494 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:08.176588 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:04:08.176899 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:08.177601 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:08.178742 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:08.179164 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:08.179316 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:08.179373 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:08.179608 (+ 235us) server_negotiation.cc:299] Negotiation successful
0811 19:04:08.179692 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 19:04:08.180264 1133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:08.175741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.2:35775 (local address 127.13.109.1:57619)
0811 19:04:08.176171 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:08.176186 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 19:04:08.176290 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:04:08.176597 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:04:08.176600 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:04:08.176616 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:04:08.176768 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:08.176774 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:08.177746 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:08.177752 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:08.178592 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:08.178602 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:08.179313 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:08.179317 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:08.179436 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:08.179458 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 19:04:08.180022 (+ 564us) client_negotiation.cc:240] Negotiation successful
0811 19:04:08.180112 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":348,"mutex_wait_us":19,"thread_start_us":145,"threads_started":1}
I20260811 19:04:08.180269 1048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f3c8994480948e9a7e74e1616d24991" candidate_uuid: "e852feeaf434478d93a92447d91c89b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04b7bcf259a842999c22a3c0c4e4c11f" is_pre_election: true
I20260811 19:04:08.180660 1048 raft_consensus.cc:2454] T 0f3c8994480948e9a7e74e1616d24991 P 04b7bcf259a842999c22a3c0c4e4c11f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e852feeaf434478d93a92447d91c89b9 in term 0.
I20260811 19:04:08.181288 683 leader_election.cc:302] T 0f3c8994480948e9a7e74e1616d24991 P e852feeaf434478d93a92447d91c89b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04b7bcf259a842999c22a3c0c4e4c11f, e852feeaf434478d93a92447d91c89b9; no voters:
I20260811 19:04:08.181582 906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f3c8994480948e9a7e74e1616d24991" candidate_uuid: "e852feeaf434478d93a92447d91c89b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9a3bccd535d4423aadedeed463a3088" is_pre_election: true
I20260811 19:04:08.181630 1130 raft_consensus.cc:2781] T 0f3c8994480948e9a7e74e1616d24991 P e852feeaf434478d93a92447d91c89b9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 19:04:08.181727 1130 raft_consensus.cc:482] T 0f3c8994480948e9a7e74e1616d24991 P e852feeaf434478d93a92447d91c89b9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 19:04:08.181774 1130 raft_consensus.cc:3037] T 0f3c8994480948e9a7e74e1616d24991 P e852feeaf434478d93a92447d91c89b9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:04:08.181946 906 raft_consensus.cc:2454] T 0f3c8994480948e9a7e74e1616d24991 P b9a3bccd535d4423aadedeed463a3088 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e852feeaf434478d93a92447d91c89b9 in term 0.
I20260811 19:04:08.183007 1130 raft_consensus.cc:504] T 0f3c8994480948e9a7e74e1616d24991 P e852feeaf434478d93a92447d91c89b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04b7bcf259a842999c22a3c0c4e4c11f" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 34755 } } peers { permanent_uuid: "e852feeaf434478d93a92447d91c89b9" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 37477 } } peers { permanent_uuid: "b9a3bccd535d4423aadedeed463a3088" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 35775 } }
I20260811 19:04:08.183339 1130 leader_election.cc:288] T 0f3c8994480948e9a7e74e1616d24991 P e852feeaf434478d93a92447d91c89b9 [CANDIDATE]: Term 1 election: Requested vote from peers 04b7bcf259a842999c22a3c0c4e4c11f (127.13.109.3:34755), b9a3bccd535d4423aadedeed463a3088 (127.13.109.2:35775)
I20260811 19:04:08.183840 1048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f3c8994480948e9a7e74e1616d24991" candidate_uuid: "e852feeaf434478d93a92447d91c89b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04b7bcf259a842999c22a3c0c4e4c11f"
I20260811 19:04:08.183971 906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f3c8994480948e9a7e74e1616d24991" candidate_uuid: "e852feeaf434478d93a92447d91c89b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9a3bccd535d4423aadedeed463a3088"
I20260811 19:04:08.184082 906 raft_consensus.cc:3037] T 0f3c8994480948e9a7e74e1616d24991 P b9a3bccd535d4423aadedeed463a3088 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:04:08.184042 1048 raft_consensus.cc:3037] T 0f3c8994480948e9a7e74e1616d24991 P 04b7bcf259a842999c22a3c0c4e4c11f [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:04:08.185317 1048 raft_consensus.cc:2454] T 0f3c8994480948e9a7e74e1616d24991 P 04b7bcf259a842999c22a3c0c4e4c11f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e852feeaf434478d93a92447d91c89b9 in term 1.
I20260811 19:04:08.185339 906 raft_consensus.cc:2454] T 0f3c8994480948e9a7e74e1616d24991 P b9a3bccd535d4423aadedeed463a3088 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e852feeaf434478d93a92447d91c89b9 in term 1.
I20260811 19:04:08.185691 683 leader_election.cc:302] T 0f3c8994480948e9a7e74e1616d24991 P e852feeaf434478d93a92447d91c89b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04b7bcf259a842999c22a3c0c4e4c11f, e852feeaf434478d93a92447d91c89b9; no voters:
I20260811 19:04:08.185998 1130 raft_consensus.cc:2781] T 0f3c8994480948e9a7e74e1616d24991 P e852feeaf434478d93a92447d91c89b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:04:08.186291 1130 raft_consensus.cc:686] T 0f3c8994480948e9a7e74e1616d24991 P e852feeaf434478d93a92447d91c89b9 [term 1 LEADER]: Becoming Leader. State: Replica: e852feeaf434478d93a92447d91c89b9, State: Running, Role: LEADER
I20260811 19:04:08.186697 1130 consensus_queue.cc:237] T 0f3c8994480948e9a7e74e1616d24991 P e852feeaf434478d93a92447d91c89b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04b7bcf259a842999c22a3c0c4e4c11f" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 34755 } } peers { permanent_uuid: "e852feeaf434478d93a92447d91c89b9" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 37477 } } peers { permanent_uuid: "b9a3bccd535d4423aadedeed463a3088" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 35775 } }
I20260811 19:04:08.190436 597 catalog_manager.cc:5495] T 0f3c8994480948e9a7e74e1616d24991 P e852feeaf434478d93a92447d91c89b9 reported cstate change: term changed from 0 to 1, leader changed from <none> to e852feeaf434478d93a92447d91c89b9 (127.13.109.1). New cstate: current_term: 1 leader_uuid: "e852feeaf434478d93a92447d91c89b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04b7bcf259a842999c22a3c0c4e4c11f" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 34755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e852feeaf434478d93a92447d91c89b9" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 37477 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b9a3bccd535d4423aadedeed463a3088" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 35775 } health_report { overall_health: UNKNOWN } } }
I20260811 19:04:08.220775 1118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:08.216960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43830 (local address 127.13.109.1:37477)
0811 19:04:08.217145 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:08.217150 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:08.217166 (+ 16us) server_negotiation.cc:407] Connection header received
0811 19:04:08.217274 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:08.217277 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:08.217334 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:08.217464 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:04:08.217747 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:08.218287 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:08.219184 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:08.219408 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:08.219540 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:08.219910 (+ 370us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:04:08.220358 (+ 448us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:04:08.220517 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:08.220555 (+ 38us) server_negotiation.cc:299] Negotiation successful
0811 19:04:08.220610 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260811 19:04:08.227520 906 raft_consensus.cc:1261] T 0f3c8994480948e9a7e74e1616d24991 P b9a3bccd535d4423aadedeed463a3088 [term 1 FOLLOWER]: Refusing update from remote peer e852feeaf434478d93a92447d91c89b9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:04:08.227520 1048 raft_consensus.cc:1261] T 0f3c8994480948e9a7e74e1616d24991 P 04b7bcf259a842999c22a3c0c4e4c11f [term 1 FOLLOWER]: Refusing update from remote peer e852feeaf434478d93a92447d91c89b9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:04:08.228425 1130 consensus_queue.cc:1035] T 0f3c8994480948e9a7e74e1616d24991 P e852feeaf434478d93a92447d91c89b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "04b7bcf259a842999c22a3c0c4e4c11f" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 34755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:04:08.228648 1134 consensus_queue.cc:1035] T 0f3c8994480948e9a7e74e1616d24991 P e852feeaf434478d93a92447d91c89b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b9a3bccd535d4423aadedeed463a3088" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 35775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:04:08.237147 1137 mvcc.cc:205] Tried to move back new op lower bound from 7317401797530468352 to 7317401797376143360. Current Snapshot: MvccSnapshot[applied={T|T < 7317401797530468352}]
I20260811 19:04:08.239084 1139 mvcc.cc:205] Tried to move back new op lower bound from 7317401797530468352 to 7317401797376143360. Current Snapshot: MvccSnapshot[applied={T|T < 7317401797530468352}]
I20260811 19:04:08.239329 1138 mvcc.cc:205] Tried to move back new op lower bound from 7317401797530468352 to 7317401797376143360. Current Snapshot: MvccSnapshot[applied={T|T < 7317401797530468352}]
I20260811 19:04:08.248278 597 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49122:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:04:08.248512 597 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49122:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:04:08.251345 597 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9b724fe5c5e145fd8e5e3152bb5b1165: Sending DeleteTablet for 3 replicas of tablet 0f3c8994480948e9a7e74e1616d24991
I20260811 19:04:08.252425 735 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f3c8994480948e9a7e74e1616d24991 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:04:08 UTC) from {username='slave'} at 127.0.0.1:43824
I20260811 19:04:08.252429 1028 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f3c8994480948e9a7e74e1616d24991 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:04:08 UTC) from {username='slave'} at 127.0.0.1:40980
I20260811 19:04:08.252614 883 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f3c8994480948e9a7e74e1616d24991 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:04:08 UTC) from {username='slave'} at 127.0.0.1:38782
I20260811 19:04:08.253062 1148 tablet_replica.cc:330] T 0f3c8994480948e9a7e74e1616d24991 P e852feeaf434478d93a92447d91c89b9: stopping tablet replica
I20260811 19:04:08.253489 1148 raft_consensus.cc:2229] T 0f3c8994480948e9a7e74e1616d24991 P e852feeaf434478d93a92447d91c89b9 [term 1 LEADER]: Raft consensus shutting down.
I20260811 19:04:08.253590 1147 tablet_replica.cc:330] T 0f3c8994480948e9a7e74e1616d24991 P 04b7bcf259a842999c22a3c0c4e4c11f: stopping tablet replica
I20260811 19:04:08.253940 1147 raft_consensus.cc:2229] T 0f3c8994480948e9a7e74e1616d24991 P 04b7bcf259a842999c22a3c0c4e4c11f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:04:08.254024 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 665
I20260811 19:04:08.255224 1147 raft_consensus.cc:2258] T 0f3c8994480948e9a7e74e1616d24991 P 04b7bcf259a842999c22a3c0c4e4c11f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:04:08.257018 1147 ts_tablet_manager.cc:1905] T 0f3c8994480948e9a7e74e1616d24991 P 04b7bcf259a842999c22a3c0c4e4c11f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:04:08.258044 1150 tablet_replica.cc:330] T 0f3c8994480948e9a7e74e1616d24991 P b9a3bccd535d4423aadedeed463a3088: stopping tablet replica
I20260811 19:04:08.258414 1150 raft_consensus.cc:2229] T 0f3c8994480948e9a7e74e1616d24991 P b9a3bccd535d4423aadedeed463a3088 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:04:08.258761 1150 raft_consensus.cc:2258] T 0f3c8994480948e9a7e74e1616d24991 P b9a3bccd535d4423aadedeed463a3088 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:04:08.260315 1150 ts_tablet_manager.cc:1905] T 0f3c8994480948e9a7e74e1616d24991 P b9a3bccd535d4423aadedeed463a3088: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:04:08.260902 1147 ts_tablet_manager.cc:1918] T 0f3c8994480948e9a7e74e1616d24991 P 04b7bcf259a842999c22a3c0c4e4c11f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:04:08.261021 1147 log.cc:1196] T 0f3c8994480948e9a7e74e1616d24991 P 04b7bcf259a842999c22a3c0c4e4c11f: Deleting WAL directory at /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474792496492-13748-0/minicluster-data/ts-2/wal/wals/0f3c8994480948e9a7e74e1616d24991
I20260811 19:04:08.261413 1147 ts_tablet_manager.cc:1939] T 0f3c8994480948e9a7e74e1616d24991 P 04b7bcf259a842999c22a3c0c4e4c11f: Deleting consensus metadata
I20260811 19:04:08.263055 1150 ts_tablet_manager.cc:1918] T 0f3c8994480948e9a7e74e1616d24991 P b9a3bccd535d4423aadedeed463a3088: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:04:08.263005 579 catalog_manager.cc:4841] TS 04b7bcf259a842999c22a3c0c4e4c11f (127.13.109.3:34755): tablet 0f3c8994480948e9a7e74e1616d24991 (table test-table [id=1d3e7f5edc1d4c63a5fee272d1de2e55]) successfully deleted
I20260811 19:04:08.263211 1150 log.cc:1196] T 0f3c8994480948e9a7e74e1616d24991 P b9a3bccd535d4423aadedeed463a3088: Deleting WAL directory at /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474792496492-13748-0/minicluster-data/ts-1/wal/wals/0f3c8994480948e9a7e74e1616d24991
I20260811 19:04:08.263523 1150 ts_tablet_manager.cc:1939] T 0f3c8994480948e9a7e74e1616d24991 P b9a3bccd535d4423aadedeed463a3088: Deleting consensus metadata
I20260811 19:04:08.264766 579 catalog_manager.cc:4841] TS b9a3bccd535d4423aadedeed463a3088 (127.13.109.2:35775): tablet 0f3c8994480948e9a7e74e1616d24991 (table test-table [id=1d3e7f5edc1d4c63a5fee272d1de2e55]) successfully deleted
W20260811 19:04:08.267222 580 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.109.1:37477 (error 108)
I20260811 19:04:08.267534 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 814
I20260811 19:04:08.268599 1120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:08.268257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.1:37477 (local address 127.0.0.1:43846)
0811 19:04:08.268415 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:08.268495 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.1:37477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260811 19:04:08.268841 580 catalog_manager.cc:4584] TS e852feeaf434478d93a92447d91c89b9 (127.13.109.1:37477): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0f3c8994480948e9a7e74e1616d24991: Network error: Client connection negotiation failed: client connection to 127.13.109.1:37477: connect: Connection refused (error 111)
I20260811 19:04:08.276330 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 959
I20260811 19:04:08.286198 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 559
2026-08-11T19:04:08Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1197 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:04:08 dist-test-slave-0kls krb5kdc[1154](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:04:08 dist-test-slave-0kls krb5kdc[1154](info): set up 2 sockets
Aug 11 19:04:08 dist-test-slave-0kls krb5kdc[1154](info): commencing operation
krb5kdc: starting...
W20260811 19:04:09.396230 13748 mini_kdc.cc:121] Time spent starting KDC: real 1.085s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/krb5kdc/test-user.keytab.
Aug 11 19:04:09 dist-test-slave-0kls krb5kdc[1154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475049, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:04:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:04:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.109.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.109.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.109.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab.
Entry for principal kudu/127.13.109.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab.
I20260811 19:04:09.544958 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.62:33699
--webserver_interface=127.13.109.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab
--principal=kudu/127.13.109.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:44777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.109.62:33699
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:04:09.678408 1168 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:04:09.678838 1168 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:04:09.678937 1168 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:04:09.684073 1168 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:04:09.684222 1168 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:04:09.684263 1168 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:04:09.684301 1168 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:04:09.684341 1168 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:04:09.690582 1168 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:44777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.109.62:33699
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab
--principal=kudu/127.13.109.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.62:33699
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.109.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1168
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:04:09.692028 1168 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:04:09.693214 1168 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:04:09.702936 1178 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:04:09.702960 1175 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:04:09.703047 1168 server_base.cc:1034] running on GCE node
W20260811 19:04:09.702960 1176 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:04:09.704016 1168 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:04:09.705353 1168 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:04:09.706579 1168 hybrid_clock.cc:648] HybridClock initialized: now 1786475049706569 us; error 32 us; skew 500 ppm
Aug 11 19:04:09 dist-test-slave-0kls krb5kdc[1154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475049, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:04:09.714818 1168 init.cc:377] Logged in from keytab as kudu/127.13.109.62@KRBTEST.COM (short username kudu)
I20260811 19:04:09.715936 1168 webserver.cc:460] Webserver started at http://127.13.109.62:46091/ using document root <none> and password file <none>
I20260811 19:04:09.716616 1168 fs_manager.cc:359] Metadata directory not provided
I20260811 19:04:09.716729 1168 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:04:09.717006 1168 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:04:09.719533 1168 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/minicluster-data/master-0/data/instance:
uuid: "5da8dcdb2b1746b790ec6c8e29d037b5"
format_stamp: "Formatted at 2026-08-11 19:04:09 on dist-test-slave-0kls"
server_key: "fb6b130bc85923170989ec8051a6b367"
server_key_iv: "95d9972200c329c9b9e5e91d6126034d"
server_key_version: "encryptionkey@0"
I20260811 19:04:09.720247 1168 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/minicluster-data/master-0/wal/instance:
uuid: "5da8dcdb2b1746b790ec6c8e29d037b5"
format_stamp: "Formatted at 2026-08-11 19:04:09 on dist-test-slave-0kls"
server_key: "fb6b130bc85923170989ec8051a6b367"
server_key_iv: "95d9972200c329c9b9e5e91d6126034d"
server_key_version: "encryptionkey@0"
I20260811 19:04:09.724623 1168 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 19:04:09.727516 1185 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:04:09.728981 1168 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 19:04:09.729172 1168 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/minicluster-data/master-0/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/minicluster-data/master-0/wal
uuid: "5da8dcdb2b1746b790ec6c8e29d037b5"
format_stamp: "Formatted at 2026-08-11 19:04:09 on dist-test-slave-0kls"
server_key: "fb6b130bc85923170989ec8051a6b367"
server_key_iv: "95d9972200c329c9b9e5e91d6126034d"
server_key_version: "encryptionkey@0"
I20260811 19:04:09.729327 1168 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:04:09.744021 1168 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:04:09.747469 1168 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:04:09.747715 1168 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:04:09.756098 1168 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.62:33699
I20260811 19:04:09.756129 1243 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.62:33699 every 8 connection(s)
I20260811 19:04:09.757191 1168 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
I20260811 19:04:09.760632 1245 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:04:09.764127 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 1168
I20260811 19:04:09.764294 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/minicluster-data/master-0/wal/instance
I20260811 19:04:09.764643 13748 external_mini_cluster.cc:1428] Setting key d1413921e273093d23a3c6aa7b8c994d
I20260811 19:04:09.767746 1245 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5da8dcdb2b1746b790ec6c8e29d037b5: Bootstrap starting.
I20260811 19:04:09.770201 1245 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5da8dcdb2b1746b790ec6c8e29d037b5: Neither blocks nor log segments found. Creating new log.
I20260811 19:04:09.771023 1245 log.cc:824] T 00000000000000000000000000000000 P 5da8dcdb2b1746b790ec6c8e29d037b5: Log is configured to *not* fsync() on all Append() calls
I20260811 19:04:09.773422 1245 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5da8dcdb2b1746b790ec6c8e29d037b5: No bootstrap required, opened a new log
Aug 11 19:04:09 dist-test-slave-0kls krb5kdc[1154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475049, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
I20260811 19:04:09.777376 1245 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5da8dcdb2b1746b790ec6c8e29d037b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5da8dcdb2b1746b790ec6c8e29d037b5" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 33699 } }
I20260811 19:04:09.777694 1245 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5da8dcdb2b1746b790ec6c8e29d037b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:04:09.777779 1245 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5da8dcdb2b1746b790ec6c8e29d037b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5da8dcdb2b1746b790ec6c8e29d037b5, State: Initialized, Role: FOLLOWER
I20260811 19:04:09.778326 1245 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5da8dcdb2b1746b790ec6c8e29d037b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5da8dcdb2b1746b790ec6c8e29d037b5" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 33699 } }
I20260811 19:04:09.778484 1245 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5da8dcdb2b1746b790ec6c8e29d037b5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:04:09.778564 1245 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5da8dcdb2b1746b790ec6c8e29d037b5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:04:09.778668 1245 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5da8dcdb2b1746b790ec6c8e29d037b5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:04:09.779663 1245 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5da8dcdb2b1746b790ec6c8e29d037b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5da8dcdb2b1746b790ec6c8e29d037b5" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 33699 } }
I20260811 19:04:09.780042 1245 leader_election.cc:302] T 00000000000000000000000000000000 P 5da8dcdb2b1746b790ec6c8e29d037b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5da8dcdb2b1746b790ec6c8e29d037b5; no voters:
I20260811 19:04:09.780395 1245 leader_election.cc:288] T 00000000000000000000000000000000 P 5da8dcdb2b1746b790ec6c8e29d037b5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:04:09.780548 1251 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5da8dcdb2b1746b790ec6c8e29d037b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:04:09.781061 1251 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5da8dcdb2b1746b790ec6c8e29d037b5 [term 1 LEADER]: Becoming Leader. State: Replica: 5da8dcdb2b1746b790ec6c8e29d037b5, State: Running, Role: LEADER
I20260811 19:04:09.781482 1251 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5da8dcdb2b1746b790ec6c8e29d037b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5da8dcdb2b1746b790ec6c8e29d037b5" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 33699 } }
I20260811 19:04:09.781929 1245 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5da8dcdb2b1746b790ec6c8e29d037b5 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:04:09.782039 1248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:09.766365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56424 (local address 127.13.109.62:33699)
0811 19:04:09.766891 (+ 526us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:09.766907 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:09.766946 (+ 39us) server_negotiation.cc:407] Connection header received
0811 19:04:09.767613 (+ 667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:09.767630 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:09.768045 (+ 415us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:09.768406 (+ 361us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:09.769270 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:09.770447 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:09.771375 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:09.772051 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:09.772256 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:09.775853 (+ 3597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:04:09.775859 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:04:09.775875 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:04:09.775908 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:04:09.778357 (+ 2449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:09.778928 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:09.778941 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:09.778946 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:09.779066 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:09.779370 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:09.779374 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:09.779376 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:09.779876 (+ 500us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:04:09.780111 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:09.780648 (+ 537us) server_negotiation.cc:299] Negotiation successful
0811 19:04:09.781001 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":186,"threads_started":1}
I20260811 19:04:09.782866 1252 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5da8dcdb2b1746b790ec6c8e29d037b5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5da8dcdb2b1746b790ec6c8e29d037b5. Latest consensus state: current_term: 1 leader_uuid: "5da8dcdb2b1746b790ec6c8e29d037b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5da8dcdb2b1746b790ec6c8e29d037b5" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 33699 } } }
I20260811 19:04:09.782989 1252 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5da8dcdb2b1746b790ec6c8e29d037b5 [sys.catalog]: This master's current role is: LEADER
I20260811 19:04:09.783710 1253 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5da8dcdb2b1746b790ec6c8e29d037b5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5da8dcdb2b1746b790ec6c8e29d037b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5da8dcdb2b1746b790ec6c8e29d037b5" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 33699 } } }
I20260811 19:04:09.783806 1253 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5da8dcdb2b1746b790ec6c8e29d037b5 [sys.catalog]: This master's current role is: LEADER
I20260811 19:04:09.784647 1260 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:04:09.787886 1260 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:04:09.794728 1260 catalog_manager.cc:1347] Generated new cluster ID: 2a346a63bfc447a5b10a1a51ee79e2eb
I20260811 19:04:09.794821 1260 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:04:09.802261 1260 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:04:09.803406 1260 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:04:09.809047 1260 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5da8dcdb2b1746b790ec6c8e29d037b5: Generated new TSK 0
I20260811 19:04:09.809971 1260 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.109.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.109.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.109.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.1.keytab.
Entry for principal kudu/127.13.109.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.1.keytab.
I20260811 19:04:09.842900 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.1:0
--local_ip_for_outbound_sockets=127.13.109.1
--webserver_interface=127.13.109.1
--webserver_port=0
--tserver_master_addrs=127.13.109.62:33699
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.1.keytab
--principal=kudu/127.13.109.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:44777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:04:09.967229 1275 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:04:09.967537 1275 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:04:09.967585 1275 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:04:09.971972 1275 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:04:09.972046 1275 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:04:09.972153 1275 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.1
I20260811 19:04:09.977658 1275 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:44777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.1.keytab
--principal=kudu/127.13.109.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.109.1
--webserver_port=0
--tserver_master_addrs=127.13.109.62:33699
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1275
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.1
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:04:09.978863 1275 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:04:09.980222 1275 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:04:09.990661 1281 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:04:09.990693 1280 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:04:09.990661 1283 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:04:09.990880 1275 server_base.cc:1034] running on GCE node
I20260811 19:04:09.991557 1275 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:04:09.992295 1275 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:04:09.993548 1275 hybrid_clock.cc:648] HybridClock initialized: now 1786475049993527 us; error 35 us; skew 500 ppm
Aug 11 19:04:09 dist-test-slave-0kls krb5kdc[1154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475049, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:04:09.996519 1275 init.cc:377] Logged in from keytab as kudu/127.13.109.1@KRBTEST.COM (short username kudu)
I20260811 19:04:09.997504 1275 webserver.cc:460] Webserver started at http://127.13.109.1:45469/ using document root <none> and password file <none>
I20260811 19:04:09.998188 1275 fs_manager.cc:359] Metadata directory not provided
I20260811 19:04:09.998286 1275 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:04:09.998534 1275 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:04:10.000715 1275 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/minicluster-data/ts-0/data/instance:
uuid: "693e6dd7718e4820b474b39d1149404b"
format_stamp: "Formatted at 2026-08-11 19:04:09 on dist-test-slave-0kls"
server_key: "e2fcf2049a32431448a84b06cae7b2e3"
server_key_iv: "61e2f333d4477c63072fd6768d4ce1dc"
server_key_version: "encryptionkey@0"
I20260811 19:04:10.001296 1275 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/minicluster-data/ts-0/wal/instance:
uuid: "693e6dd7718e4820b474b39d1149404b"
format_stamp: "Formatted at 2026-08-11 19:04:09 on dist-test-slave-0kls"
server_key: "e2fcf2049a32431448a84b06cae7b2e3"
server_key_iv: "61e2f333d4477c63072fd6768d4ce1dc"
server_key_version: "encryptionkey@0"
I20260811 19:04:10.006839 1275 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260811 19:04:10.010074 1290 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:04:10.012607 1275 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 19:04:10.012775 1275 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/minicluster-data/ts-0/wal
uuid: "693e6dd7718e4820b474b39d1149404b"
format_stamp: "Formatted at 2026-08-11 19:04:09 on dist-test-slave-0kls"
server_key: "e2fcf2049a32431448a84b06cae7b2e3"
server_key_iv: "61e2f333d4477c63072fd6768d4ce1dc"
server_key_version: "encryptionkey@0"
I20260811 19:04:10.012904 1275 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:04:10.030331 1275 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:04:10.034126 1275 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:04:10.034403 1275 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:04:10.035153 1275 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:04:10.036162 1275 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:04:10.036221 1275 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:04:10.036296 1275 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:04:10.036331 1275 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:04:10.046298 1275 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.1:35957
I20260811 19:04:10.046319 1419 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.1:35957 every 8 connection(s)
I20260811 19:04:10.047287 1275 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
I20260811 19:04:10.051575 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 1275
I20260811 19:04:10.051729 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/minicluster-data/ts-0/wal/instance
I20260811 19:04:10.052116 13748 external_mini_cluster.cc:1428] Setting key c8d6d82eb018693e6282612ce0cd98c9
Aug 11 19:04:10 dist-test-slave-0kls krb5kdc[1154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475049, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.1@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
I20260811 19:04:10.063184 1248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:10.049199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.1:56481 (local address 127.13.109.62:33699)
0811 19:04:10.049409 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:10.049415 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:10.050321 (+ 906us) server_negotiation.cc:407] Connection header received
0811 19:04:10.051760 (+ 1439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:10.051763 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:10.051830 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:10.051939 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:10.053484 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:10.054321 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:10.055513 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:10.055908 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:10.056044 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:10.059005 (+ 2961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:04:10.059013 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:04:10.059017 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:04:10.059052 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:04:10.060783 (+ 1731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:10.061416 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:10.061420 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:10.061421 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:10.061539 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:10.061985 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:10.061990 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:10.061991 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:10.062200 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:04:10.062316 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:10.062865 (+ 549us) server_negotiation.cc:299] Negotiation successful
0811 19:04:10.063007 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
WARNING: no policy specified for kudu/127.13.109.2@KRBTEST.COM; defaulting to no policy
I20260811 19:04:10.064137 1422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:10.049611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:33699 (local address 127.13.109.1:56481)
0811 19:04:10.050157 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:10.050210 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 19:04:10.051504 (+ 1294us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:04:10.052105 (+ 601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:04:10.052112 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:04:10.052572 (+ 460us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:04:10.053265 (+ 693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:10.053286 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:10.054466 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:10.054473 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:10.055383 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:10.055393 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:10.056092 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:10.056098 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:10.056281 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:10.056943 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:04:10.056961 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:04:10.058796 (+ 1835us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:04:10.060924 (+ 2128us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:04:10.060931 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:04:10.060956 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:04:10.061298 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:04:10.061650 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:04:10.061654 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:04:10.061656 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:04:10.061883 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:04:10.062315 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:04:10.062324 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:04:10.062608 (+ 284us) client_negotiation.cc:769] Sending connection context
0811 19:04:10.062806 (+ 198us) client_negotiation.cc:240] Negotiation successful
0811 19:04:10.063182 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":193,"threads_started":1}
I20260811 19:04:10.065641 1420 heartbeater.cc:347] Connected to a master server at 127.13.109.62:33699
I20260811 19:04:10.065992 1420 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.109.2@KRBTEST.COM" created.
I20260811 19:04:10.066643 1420 heartbeater.cc:511] Master 127.13.109.62:33699 requested a full tablet report, sending...
I20260811 19:04:10.068517 1205 ts_manager.cc:194] Registered new tserver with Master: 693e6dd7718e4820b474b39d1149404b (127.13.109.1:35957)
I20260811 19:04:10.069772 1205 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.109.1@KRBTEST.COM'} at 127.13.109.1:56481
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.109.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.2.keytab.
Entry for principal kudu/127.13.109.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.2.keytab.
I20260811 19:04:10.086141 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.2:0
--local_ip_for_outbound_sockets=127.13.109.2
--webserver_interface=127.13.109.2
--webserver_port=0
--tserver_master_addrs=127.13.109.62:33699
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.2.keytab
--principal=kudu/127.13.109.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:44777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:04:10.207914 1425 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:04:10.208199 1425 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:04:10.208245 1425 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:04:10.212287 1425 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:04:10.212358 1425 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:04:10.212419 1425 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.2
I20260811 19:04:10.217979 1425 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:44777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.2.keytab
--principal=kudu/127.13.109.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.109.2
--webserver_port=0
--tserver_master_addrs=127.13.109.62:33699
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1425
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.2
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:04:10.219206 1425 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:04:10.220270 1425 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:04:10.230647 1431 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:04:10.230566 1433 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:04:10.230805 1425 server_base.cc:1034] running on GCE node
W20260811 19:04:10.230572 1430 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:04:10.231380 1425 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:04:10.232167 1425 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:04:10.233372 1425 hybrid_clock.cc:648] HybridClock initialized: now 1786475050233361 us; error 29 us; skew 500 ppm
Aug 11 19:04:10 dist-test-slave-0kls krb5kdc[1154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475050, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:04:10.236474 1425 init.cc:377] Logged in from keytab as kudu/127.13.109.2@KRBTEST.COM (short username kudu)
I20260811 19:04:10.237416 1425 webserver.cc:460] Webserver started at http://127.13.109.2:45281/ using document root <none> and password file <none>
I20260811 19:04:10.238096 1425 fs_manager.cc:359] Metadata directory not provided
I20260811 19:04:10.238211 1425 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:04:10.238463 1425 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:04:10.240639 1425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/minicluster-data/ts-1/data/instance:
uuid: "daa14490267b433f83f67f40a8d8367c"
format_stamp: "Formatted at 2026-08-11 19:04:10 on dist-test-slave-0kls"
server_key: "69f955a9c7b240030bcf54cb96c03d24"
server_key_iv: "4eb2a54cccecd57dc74dd793c215051c"
server_key_version: "encryptionkey@0"
I20260811 19:04:10.241228 1425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/minicluster-data/ts-1/wal/instance:
uuid: "daa14490267b433f83f67f40a8d8367c"
format_stamp: "Formatted at 2026-08-11 19:04:10 on dist-test-slave-0kls"
server_key: "69f955a9c7b240030bcf54cb96c03d24"
server_key_iv: "4eb2a54cccecd57dc74dd793c215051c"
server_key_version: "encryptionkey@0"
I20260811 19:04:10.245208 1425 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.002s
I20260811 19:04:10.247931 1443 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:04:10.249269 1425 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 19:04:10.249426 1425 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/minicluster-data/ts-1/wal
uuid: "daa14490267b433f83f67f40a8d8367c"
format_stamp: "Formatted at 2026-08-11 19:04:10 on dist-test-slave-0kls"
server_key: "69f955a9c7b240030bcf54cb96c03d24"
server_key_iv: "4eb2a54cccecd57dc74dd793c215051c"
server_key_version: "encryptionkey@0"
I20260811 19:04:10.249558 1425 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:04:10.281809 1425 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:04:10.285681 1425 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:04:10.285921 1425 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:04:10.286559 1425 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:04:10.287420 1425 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:04:10.287494 1425 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:04:10.287566 1425 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:04:10.287616 1425 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:04:10.296423 1425 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.2:44447
I20260811 19:04:10.296444 1570 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.2:44447 every 8 connection(s)
I20260811 19:04:10.297333 1425 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
I20260811 19:04:10.304656 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 1425
I20260811 19:04:10.304816 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/minicluster-data/ts-1/wal/instance
I20260811 19:04:10.305223 13748 external_mini_cluster.cc:1428] Setting key 43d37f83ed986a2921e57ee1bcea170e
Aug 11 19:04:10 dist-test-slave-0kls krb5kdc[1154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475050, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.2@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
I20260811 19:04:10.311832 1248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:10.299179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.2:60587 (local address 127.13.109.62:33699)
0811 19:04:10.299335 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:10.299339 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:10.300215 (+ 876us) server_negotiation.cc:407] Connection header received
0811 19:04:10.301559 (+ 1344us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:10.301562 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:10.301621 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:10.301732 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:10.302900 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:10.303439 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:10.304576 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:10.304870 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:10.304996 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:10.307838 (+ 2842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:04:10.307842 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:04:10.307844 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:04:10.307867 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:04:10.309345 (+ 1478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:10.310063 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:10.310066 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:10.310068 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:10.310150 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:10.310559 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:10.310562 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:10.310563 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:10.310755 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:04:10.310907 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:10.311446 (+ 539us) server_negotiation.cc:299] Negotiation successful
0811 19:04:10.311630 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 19:04:10.312647 1574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:10.299560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:33699 (local address 127.13.109.2:60587)
0811 19:04:10.300059 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:10.300105 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 19:04:10.301351 (+ 1246us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:04:10.301928 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:04:10.301935 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:04:10.302278 (+ 343us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:04:10.302748 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:10.302762 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:10.303575 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:10.303583 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:10.304440 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:10.304449 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:10.305077 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:10.305082 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:10.305309 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:10.306010 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:04:10.306026 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:04:10.307636 (+ 1610us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:04:10.309485 (+ 1849us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:04:10.309493 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:04:10.309513 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:04:10.309942 (+ 429us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:04:10.310251 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:04:10.310254 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:04:10.310258 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:04:10.310451 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:04:10.310904 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:04:10.310912 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:04:10.311175 (+ 263us) client_negotiation.cc:769] Sending connection context
0811 19:04:10.311383 (+ 208us) client_negotiation.cc:240] Negotiation successful
0811 19:04:10.311760 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":143,"threads_started":1}
I20260811 19:04:10.314199 1571 heartbeater.cc:347] Connected to a master server at 127.13.109.62:33699
I20260811 19:04:10.314478 1571 heartbeater.cc:464] Registering TS with master...
I20260811 19:04:10.315042 1571 heartbeater.cc:511] Master 127.13.109.62:33699 requested a full tablet report, sending...
I20260811 19:04:10.316264 1205 ts_manager.cc:194] Registered new tserver with Master: daa14490267b433f83f67f40a8d8367c (127.13.109.2:44447)
I20260811 19:04:10.316923 1205 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.109.2@KRBTEST.COM'} at 127.13.109.2:60587
WARNING: no policy specified for kudu/127.13.109.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.109.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.109.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.3.keytab.
Entry for principal kudu/127.13.109.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.3.keytab.
I20260811 19:04:10.336515 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.3:0
--local_ip_for_outbound_sockets=127.13.109.3
--webserver_interface=127.13.109.3
--webserver_port=0
--tserver_master_addrs=127.13.109.62:33699
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.3.keytab
--principal=kudu/127.13.109.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:44777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:04:10.455977 1577 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:04:10.456344 1577 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:04:10.456457 1577 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:04:10.460867 1577 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:04:10.460996 1577 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:04:10.461100 1577 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.3
I20260811 19:04:10.466960 1577 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:44777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.3.keytab
--principal=kudu/127.13.109.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.109.3
--webserver_port=0
--tserver_master_addrs=127.13.109.62:33699
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1577
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.3
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:04:10.468293 1577 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:04:10.469465 1577 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:04:10.480130 1583 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:04:10.480149 1582 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:04:10.480149 1585 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:04:10.480270 1577 server_base.cc:1034] running on GCE node
I20260811 19:04:10.481056 1577 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:04:10.482016 1577 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:04:10.483273 1577 hybrid_clock.cc:648] HybridClock initialized: now 1786475050483245 us; error 39 us; skew 500 ppm
Aug 11 19:04:10 dist-test-slave-0kls krb5kdc[1154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475050, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:04:10.486593 1577 init.cc:377] Logged in from keytab as kudu/127.13.109.3@KRBTEST.COM (short username kudu)
I20260811 19:04:10.487643 1577 webserver.cc:460] Webserver started at http://127.13.109.3:41669/ using document root <none> and password file <none>
I20260811 19:04:10.488328 1577 fs_manager.cc:359] Metadata directory not provided
I20260811 19:04:10.488442 1577 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:04:10.488740 1577 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:04:10.491175 1577 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/minicluster-data/ts-2/data/instance:
uuid: "9302454b5cb3465b9872246e86b59e5b"
format_stamp: "Formatted at 2026-08-11 19:04:10 on dist-test-slave-0kls"
server_key: "394a9cfc6e283feea52620d673ad0171"
server_key_iv: "b52fd5267e165b0c7c7fb07f3fac5f63"
server_key_version: "encryptionkey@0"
I20260811 19:04:10.491879 1577 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/minicluster-data/ts-2/wal/instance:
uuid: "9302454b5cb3465b9872246e86b59e5b"
format_stamp: "Formatted at 2026-08-11 19:04:10 on dist-test-slave-0kls"
server_key: "394a9cfc6e283feea52620d673ad0171"
server_key_iv: "b52fd5267e165b0c7c7fb07f3fac5f63"
server_key_version: "encryptionkey@0"
I20260811 19:04:10.496527 1577 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 19:04:10.499804 1594 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:04:10.501379 1577 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 19:04:10.501554 1577 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/minicluster-data/ts-2/wal
uuid: "9302454b5cb3465b9872246e86b59e5b"
format_stamp: "Formatted at 2026-08-11 19:04:10 on dist-test-slave-0kls"
server_key: "394a9cfc6e283feea52620d673ad0171"
server_key_iv: "b52fd5267e165b0c7c7fb07f3fac5f63"
server_key_version: "encryptionkey@0"
I20260811 19:04:10.501699 1577 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:04:10.525154 1577 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:04:10.528627 1577 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:04:10.528915 1577 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:04:10.529641 1577 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:04:10.530632 1577 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:04:10.530710 1577 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:04:10.530787 1577 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:04:10.530824 1577 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:04:10.540500 1577 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.3:37433
I20260811 19:04:10.540521 1721 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.3:37433 every 8 connection(s)
I20260811 19:04:10.541465 1577 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
I20260811 19:04:10.545033 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 1577
I20260811 19:04:10.545192 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/minicluster-data/ts-2/wal/instance
I20260811 19:04:10.545521 13748 external_mini_cluster.cc:1428] Setting key 1360b6d6440215c48f0c0afc59872b5b
Aug 11 19:04:10 dist-test-slave-0kls krb5kdc[1154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475050, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.3@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
I20260811 19:04:10.555924 1248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:10.543488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.3:37315 (local address 127.13.109.62:33699)
0811 19:04:10.543660 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:10.543665 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:10.544601 (+ 936us) server_negotiation.cc:407] Connection header received
0811 19:04:10.546182 (+ 1581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:10.546185 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:10.546250 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:10.546336 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:10.547746 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:10.548360 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:10.549470 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:10.549674 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:10.549759 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:10.552215 (+ 2456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:04:10.552218 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:04:10.552220 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:04:10.552249 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:04:10.553869 (+ 1620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:10.554519 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:10.554522 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:10.554524 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:10.554607 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:10.554921 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:10.554923 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:10.554924 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:10.555112 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:04:10.555188 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:10.555689 (+ 501us) server_negotiation.cc:299] Negotiation successful
0811 19:04:10.555799 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 19:04:10.556784 1725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:10.543942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:33699 (local address 127.13.109.3:37315)
0811 19:04:10.544432 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:10.544476 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 19:04:10.545930 (+ 1454us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:04:10.546545 (+ 615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:04:10.546576 (+ 31us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:04:10.546992 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:04:10.547590 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:10.547604 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:10.548477 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:10.548480 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:10.549361 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:10.549370 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:10.549850 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:10.549857 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:10.550012 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:10.550569 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:04:10.550585 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:04:10.552055 (+ 1470us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:04:10.553992 (+ 1937us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:04:10.553997 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:04:10.554011 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:04:10.554422 (+ 411us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:04:10.554693 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:04:10.554697 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:04:10.554698 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:04:10.554836 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:04:10.555213 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:04:10.555219 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:04:10.555464 (+ 245us) client_negotiation.cc:769] Sending connection context
0811 19:04:10.555627 (+ 163us) client_negotiation.cc:240] Negotiation successful
0811 19:04:10.555904 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":125,"threads_started":1}
I20260811 19:04:10.558336 1723 heartbeater.cc:347] Connected to a master server at 127.13.109.62:33699
I20260811 19:04:10.558610 1723 heartbeater.cc:464] Registering TS with master...
I20260811 19:04:10.559217 1723 heartbeater.cc:511] Master 127.13.109.62:33699 requested a full tablet report, sending...
I20260811 19:04:10.560330 1205 ts_manager.cc:194] Registered new tserver with Master: 9302454b5cb3465b9872246e86b59e5b (127.13.109.3:37433)
I20260811 19:04:10.560925 1205 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.109.3@KRBTEST.COM'} at 127.13.109.3:37315
I20260811 19:04:10.571848 13748 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 19:04:10.581919 1248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:10.574003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56420 (local address 127.13.109.62:33699)
0811 19:04:10.574146 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:10.574156 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:10.574292 (+ 136us) server_negotiation.cc:407] Connection header received
0811 19:04:10.574395 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:10.574398 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:10.574442 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:10.574510 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:10.575273 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:10.575791 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:10.576587 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:10.576813 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:10.576896 (+ 83us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:10.578500 (+ 1604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:04:10.578503 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:04:10.578505 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:04:10.578530 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:04:10.580094 (+ 1564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:10.580622 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:10.580626 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:10.580627 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:10.580706 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:10.581001 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:10.581003 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:10.581004 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:10.581218 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:04:10.581332 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:10.581611 (+ 279us) server_negotiation.cc:299] Negotiation successful
0811 19:04:10.581695 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 19:04:10.585295 1205 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56420:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 19:04:10.588475 1205 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 19:04:10.603652 1737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:10.597149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.2:44447 (local address 127.0.0.1:40722)
0811 19:04:10.597716 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:10.597745 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0811 19:04:10.597871 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:04:10.599399 (+ 1528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:04:10.599401 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:04:10.599421 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:04:10.599530 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:10.599536 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:10.600923 (+ 1387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:10.600927 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:10.601946 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:10.601976 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:10.603065 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:10.603069 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:10.603204 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:10.603245 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 19:04:10.603342 (+ 97us) client_negotiation.cc:240] Negotiation successful
0811 19:04:10.603474 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":500,"mutex_wait_us":1,"thread_start_us":100,"threads_started":1}
I20260811 19:04:10.603690 1735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:10.596881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.3:37433 (local address 127.0.0.1:44324)
0811 19:04:10.597271 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:10.597326 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 19:04:10.597467 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:04:10.598665 (+ 1198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:04:10.598671 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:04:10.598695 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:04:10.598850 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:10.598856 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:10.600397 (+ 1541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:10.600401 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:10.601888 (+ 1487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:10.601901 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:10.603065 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:10.603069 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:10.603438 (+ 369us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:10.603459 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 19:04:10.603526 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 19:04:10.603584 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"mutex_wait_us":299,"thread_start_us":89,"threads_started":1}
I20260811 19:04:10.604066 1736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:10.597035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44324 (local address 127.13.109.3:37433)
0811 19:04:10.598128 (+ 1093us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:10.598144 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:10.598169 (+ 25us) server_negotiation.cc:407] Connection header received
0811 19:04:10.598262 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:10.598269 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:10.598497 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:10.598679 (+ 182us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:04:10.598970 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:10.600225 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:10.602026 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:10.602909 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:10.603118 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:10.603305 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:10.603792 (+ 487us) server_negotiation.cc:299] Negotiation successful
0811 19:04:10.603909 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":980,"thread_start_us":122,"threads_started":1}
I20260811 19:04:10.604702 1740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:10.597614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40722 (local address 127.13.109.2:44447)
0811 19:04:10.598266 (+ 652us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:10.598276 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:10.598296 (+ 20us) server_negotiation.cc:407] Connection header received
0811 19:04:10.598361 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:10.598365 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:10.598567 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:10.599402 (+ 835us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:04:10.599664 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:10.600765 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:10.602113 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:10.602909 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:10.604275 (+ 1366us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:10.604418 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:10.604467 (+ 49us) server_negotiation.cc:299] Negotiation successful
0811 19:04:10.604553 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":560,"thread_start_us":124,"threads_started":1}
I20260811 19:04:10.604740 1741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:10.597656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.1:35957 (local address 127.0.0.1:47684)
0811 19:04:10.597990 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:10.598004 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 19:04:10.598098 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:04:10.599207 (+ 1109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:04:10.599210 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:04:10.599228 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:04:10.599309 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:10.599313 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:10.600612 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:10.600618 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:10.602349 (+ 1731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:10.602360 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:10.603710 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:10.603979 (+ 269us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:10.604410 (+ 431us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:10.604435 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 19:04:10.604505 (+ 70us) client_negotiation.cc:240] Negotiation successful
0811 19:04:10.604579 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"mutex_wait_us":1,"thread_start_us":94,"threads_started":1}
I20260811 19:04:10.605273 1742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:10.597851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47684 (local address 127.13.109.1:35957)
0811 19:04:10.598749 (+ 898us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:10.598756 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:10.598779 (+ 23us) server_negotiation.cc:407] Connection header received
0811 19:04:10.598865 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:10.598870 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:10.599068 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:10.599201 (+ 133us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:04:10.599459 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:10.600465 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:10.602499 (+ 2034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:10.603373 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:10.603498 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:10.603640 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:10.605055 (+ 1415us) server_negotiation.cc:299] Negotiation successful
0811 19:04:10.605161 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":73,"threads_started":1}
I20260811 19:04:10.607486 1500 tablet_service.cc:1467] Processing CreateTablet for tablet f67886eb1bdf4e15a23c7a2c19c3bd9b (DEFAULT_TABLE table=test-table [id=692b4b47f0b24611902de399b2f4ac50]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:04:10.607488 1647 tablet_service.cc:1467] Processing CreateTablet for tablet f67886eb1bdf4e15a23c7a2c19c3bd9b (DEFAULT_TABLE table=test-table [id=692b4b47f0b24611902de399b2f4ac50]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:04:10.607707 1342 tablet_service.cc:1467] Processing CreateTablet for tablet f67886eb1bdf4e15a23c7a2c19c3bd9b (DEFAULT_TABLE table=test-table [id=692b4b47f0b24611902de399b2f4ac50]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:04:10.608686 1647 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f67886eb1bdf4e15a23c7a2c19c3bd9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:04:10.608680 1500 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f67886eb1bdf4e15a23c7a2c19c3bd9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:04:10.608696 1342 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f67886eb1bdf4e15a23c7a2c19c3bd9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:04:10.615475 1743 tablet_bootstrap.cc:492] T f67886eb1bdf4e15a23c7a2c19c3bd9b P 693e6dd7718e4820b474b39d1149404b: Bootstrap starting.
I20260811 19:04:10.616822 1745 tablet_bootstrap.cc:492] T f67886eb1bdf4e15a23c7a2c19c3bd9b P 9302454b5cb3465b9872246e86b59e5b: Bootstrap starting.
I20260811 19:04:10.616988 1744 tablet_bootstrap.cc:492] T f67886eb1bdf4e15a23c7a2c19c3bd9b P daa14490267b433f83f67f40a8d8367c: Bootstrap starting.
I20260811 19:04:10.618958 1743 tablet_bootstrap.cc:654] T f67886eb1bdf4e15a23c7a2c19c3bd9b P 693e6dd7718e4820b474b39d1149404b: Neither blocks nor log segments found. Creating new log.
I20260811 19:04:10.619325 1745 tablet_bootstrap.cc:654] T f67886eb1bdf4e15a23c7a2c19c3bd9b P 9302454b5cb3465b9872246e86b59e5b: Neither blocks nor log segments found. Creating new log.
I20260811 19:04:10.619903 1743 log.cc:824] T f67886eb1bdf4e15a23c7a2c19c3bd9b P 693e6dd7718e4820b474b39d1149404b: Log is configured to *not* fsync() on all Append() calls
I20260811 19:04:10.619971 1744 tablet_bootstrap.cc:654] T f67886eb1bdf4e15a23c7a2c19c3bd9b P daa14490267b433f83f67f40a8d8367c: Neither blocks nor log segments found. Creating new log.
I20260811 19:04:10.620844 1745 log.cc:824] T f67886eb1bdf4e15a23c7a2c19c3bd9b P 9302454b5cb3465b9872246e86b59e5b: Log is configured to *not* fsync() on all Append() calls
I20260811 19:04:10.620844 1744 log.cc:824] T f67886eb1bdf4e15a23c7a2c19c3bd9b P daa14490267b433f83f67f40a8d8367c: Log is configured to *not* fsync() on all Append() calls
I20260811 19:04:10.622537 1743 tablet_bootstrap.cc:492] T f67886eb1bdf4e15a23c7a2c19c3bd9b P 693e6dd7718e4820b474b39d1149404b: No bootstrap required, opened a new log
I20260811 19:04:10.623078 1743 ts_tablet_manager.cc:1397] T f67886eb1bdf4e15a23c7a2c19c3bd9b P 693e6dd7718e4820b474b39d1149404b: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260811 19:04:10.623142 1745 tablet_bootstrap.cc:492] T f67886eb1bdf4e15a23c7a2c19c3bd9b P 9302454b5cb3465b9872246e86b59e5b: No bootstrap required, opened a new log
I20260811 19:04:10.623142 1744 tablet_bootstrap.cc:492] T f67886eb1bdf4e15a23c7a2c19c3bd9b P daa14490267b433f83f67f40a8d8367c: No bootstrap required, opened a new log
I20260811 19:04:10.623342 1744 ts_tablet_manager.cc:1397] T f67886eb1bdf4e15a23c7a2c19c3bd9b P daa14490267b433f83f67f40a8d8367c: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 19:04:10.623342 1745 ts_tablet_manager.cc:1397] T f67886eb1bdf4e15a23c7a2c19c3bd9b P 9302454b5cb3465b9872246e86b59e5b: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 19:04:10.626881 1743 raft_consensus.cc:348] T f67886eb1bdf4e15a23c7a2c19c3bd9b P 693e6dd7718e4820b474b39d1149404b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9302454b5cb3465b9872246e86b59e5b" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 37433 } } peers { permanent_uuid: "693e6dd7718e4820b474b39d1149404b" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 35957 } } peers { permanent_uuid: "daa14490267b433f83f67f40a8d8367c" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 44447 } }
I20260811 19:04:10.627270 1743 raft_consensus.cc:374] T f67886eb1bdf4e15a23c7a2c19c3bd9b P 693e6dd7718e4820b474b39d1149404b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:04:10.627345 1743 raft_consensus.cc:729] T f67886eb1bdf4e15a23c7a2c19c3bd9b P 693e6dd7718e4820b474b39d1149404b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 693e6dd7718e4820b474b39d1149404b, State: Initialized, Role: FOLLOWER
I20260811 19:04:10.627858 1743 consensus_queue.cc:260] T f67886eb1bdf4e15a23c7a2c19c3bd9b P 693e6dd7718e4820b474b39d1149404b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9302454b5cb3465b9872246e86b59e5b" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 37433 } } peers { permanent_uuid: "693e6dd7718e4820b474b39d1149404b" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 35957 } } peers { permanent_uuid: "daa14490267b433f83f67f40a8d8367c" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 44447 } }
I20260811 19:04:10.627763 1745 raft_consensus.cc:348] T f67886eb1bdf4e15a23c7a2c19c3bd9b P 9302454b5cb3465b9872246e86b59e5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9302454b5cb3465b9872246e86b59e5b" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 37433 } } peers { permanent_uuid: "693e6dd7718e4820b474b39d1149404b" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 35957 } } peers { permanent_uuid: "daa14490267b433f83f67f40a8d8367c" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 44447 } }
I20260811 19:04:10.627763 1744 raft_consensus.cc:348] T f67886eb1bdf4e15a23c7a2c19c3bd9b P daa14490267b433f83f67f40a8d8367c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9302454b5cb3465b9872246e86b59e5b" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 37433 } } peers { permanent_uuid: "693e6dd7718e4820b474b39d1149404b" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 35957 } } peers { permanent_uuid: "daa14490267b433f83f67f40a8d8367c" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 44447 } }
I20260811 19:04:10.628072 1745 raft_consensus.cc:374] T f67886eb1bdf4e15a23c7a2c19c3bd9b P 9302454b5cb3465b9872246e86b59e5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:04:10.628072 1744 raft_consensus.cc:374] T f67886eb1bdf4e15a23c7a2c19c3bd9b P daa14490267b433f83f67f40a8d8367c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:04:10.628130 1745 raft_consensus.cc:729] T f67886eb1bdf4e15a23c7a2c19c3bd9b P 9302454b5cb3465b9872246e86b59e5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9302454b5cb3465b9872246e86b59e5b, State: Initialized, Role: FOLLOWER
I20260811 19:04:10.628141 1744 raft_consensus.cc:729] T f67886eb1bdf4e15a23c7a2c19c3bd9b P daa14490267b433f83f67f40a8d8367c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: daa14490267b433f83f67f40a8d8367c, State: Initialized, Role: FOLLOWER
I20260811 19:04:10.628609 1745 consensus_queue.cc:260] T f67886eb1bdf4e15a23c7a2c19c3bd9b P 9302454b5cb3465b9872246e86b59e5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9302454b5cb3465b9872246e86b59e5b" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 37433 } } peers { permanent_uuid: "693e6dd7718e4820b474b39d1149404b" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 35957 } } peers { permanent_uuid: "daa14490267b433f83f67f40a8d8367c" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 44447 } }
I20260811 19:04:10.628609 1744 consensus_queue.cc:260] T f67886eb1bdf4e15a23c7a2c19c3bd9b P daa14490267b433f83f67f40a8d8367c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9302454b5cb3465b9872246e86b59e5b" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 37433 } } peers { permanent_uuid: "693e6dd7718e4820b474b39d1149404b" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 35957 } } peers { permanent_uuid: "daa14490267b433f83f67f40a8d8367c" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 44447 } }
I20260811 19:04:10.629076 1743 ts_tablet_manager.cc:1428] T f67886eb1bdf4e15a23c7a2c19c3bd9b P 693e6dd7718e4820b474b39d1149404b: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260811 19:04:10.629482 1420 heartbeater.cc:503] Master 127.13.109.62:33699 was elected leader, sending a full tablet report...
I20260811 19:04:10.629678 1745 ts_tablet_manager.cc:1428] T f67886eb1bdf4e15a23c7a2c19c3bd9b P 9302454b5cb3465b9872246e86b59e5b: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 19:04:10.629678 1744 ts_tablet_manager.cc:1428] T f67886eb1bdf4e15a23c7a2c19c3bd9b P daa14490267b433f83f67f40a8d8367c: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 19:04:10.630384 1571 heartbeater.cc:503] Master 127.13.109.62:33699 was elected leader, sending a full tablet report...
I20260811 19:04:10.630643 1723 heartbeater.cc:503] Master 127.13.109.62:33699 was elected leader, sending a full tablet report...
I20260811 19:04:10.630982 1751 raft_consensus.cc:482] T f67886eb1bdf4e15a23c7a2c19c3bd9b P daa14490267b433f83f67f40a8d8367c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:04:10.631160 1751 raft_consensus.cc:504] T f67886eb1bdf4e15a23c7a2c19c3bd9b P daa14490267b433f83f67f40a8d8367c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9302454b5cb3465b9872246e86b59e5b" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 37433 } } peers { permanent_uuid: "693e6dd7718e4820b474b39d1149404b" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 35957 } } peers { permanent_uuid: "daa14490267b433f83f67f40a8d8367c" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 44447 } }
I20260811 19:04:10.635818 1751 leader_election.cc:288] T f67886eb1bdf4e15a23c7a2c19c3bd9b P daa14490267b433f83f67f40a8d8367c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9302454b5cb3465b9872246e86b59e5b (127.13.109.3:37433), 693e6dd7718e4820b474b39d1149404b (127.13.109.1:35957)
I20260811 19:04:10.638638 1574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:10.633766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.3:37433 (local address 127.13.109.2:52175)
0811 19:04:10.634038 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:10.634054 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 19:04:10.634141 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:04:10.634839 (+ 698us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:04:10.634843 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:04:10.634866 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:04:10.635013 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:10.635021 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:10.635933 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:10.635937 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:10.637387 (+ 1450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:10.637400 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:10.638262 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:10.638265 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:10.638356 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:10.638379 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 19:04:10.638449 (+ 70us) client_negotiation.cc:240] Negotiation successful
0811 19:04:10.638510 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190}
I20260811 19:04:10.638700 1736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:10.634327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.2:52175 (local address 127.13.109.3:37433)
0811 19:04:10.634483 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:10.634489 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:10.634505 (+ 16us) server_negotiation.cc:407] Connection header received
0811 19:04:10.634563 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:10.634566 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:10.634614 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:10.634708 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:04:10.635157 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:10.635789 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:10.637534 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:10.638134 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:10.638271 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:10.638329 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:10.638490 (+ 161us) server_negotiation.cc:299] Negotiation successful
0811 19:04:10.638566 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 19:04:10.639503 1668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f67886eb1bdf4e15a23c7a2c19c3bd9b" candidate_uuid: "daa14490267b433f83f67f40a8d8367c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9302454b5cb3465b9872246e86b59e5b" is_pre_election: true
I20260811 19:04:10.639932 1668 raft_consensus.cc:2454] T f67886eb1bdf4e15a23c7a2c19c3bd9b P 9302454b5cb3465b9872246e86b59e5b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate daa14490267b433f83f67f40a8d8367c in term 0.
I20260811 19:04:10.640486 1742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:10.636214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.2:43469 (local address 127.13.109.1:35957)
0811 19:04:10.636364 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:10.636370 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:10.636508 (+ 138us) server_negotiation.cc:407] Connection header received
0811 19:04:10.636588 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:10.636592 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:10.636640 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:10.636724 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:04:10.637007 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:10.637710 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:10.639239 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:10.639884 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:10.640055 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:10.640115 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:10.640294 (+ 179us) server_negotiation.cc:299] Negotiation successful
0811 19:04:10.640378 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 19:04:10.640450 1444 leader_election.cc:302] T f67886eb1bdf4e15a23c7a2c19c3bd9b P daa14490267b433f83f67f40a8d8367c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9302454b5cb3465b9872246e86b59e5b, daa14490267b433f83f67f40a8d8367c; no voters:
I20260811 19:04:10.640774 1751 raft_consensus.cc:2781] T f67886eb1bdf4e15a23c7a2c19c3bd9b P daa14490267b433f83f67f40a8d8367c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 19:04:10.640909 1751 raft_consensus.cc:482] T f67886eb1bdf4e15a23c7a2c19c3bd9b P daa14490267b433f83f67f40a8d8367c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 19:04:10.640945 1752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:10.636053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.1:35957 (local address 127.13.109.2:43469)
0811 19:04:10.636413 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:10.636425 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 19:04:10.636517 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:04:10.636776 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:04:10.636779 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:04:10.636793 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:04:10.636893 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:10.636897 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:10.637845 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:10.637848 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:10.639082 (+ 1234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:10.639092 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:10.640021 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:10.640025 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:10.640128 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:10.640158 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 19:04:10.640663 (+ 505us) client_negotiation.cc:240] Negotiation successful
0811 19:04:10.640743 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":132,"threads_started":1}
I20260811 19:04:10.640970 1751 raft_consensus.cc:3037] T f67886eb1bdf4e15a23c7a2c19c3bd9b P daa14490267b433f83f67f40a8d8367c [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:04:10.641583 1368 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f67886eb1bdf4e15a23c7a2c19c3bd9b" candidate_uuid: "daa14490267b433f83f67f40a8d8367c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "693e6dd7718e4820b474b39d1149404b" is_pre_election: true
I20260811 19:04:10.641916 1368 raft_consensus.cc:2454] T f67886eb1bdf4e15a23c7a2c19c3bd9b P 693e6dd7718e4820b474b39d1149404b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate daa14490267b433f83f67f40a8d8367c in term 0.
I20260811 19:04:10.642343 1751 raft_consensus.cc:504] T f67886eb1bdf4e15a23c7a2c19c3bd9b P daa14490267b433f83f67f40a8d8367c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9302454b5cb3465b9872246e86b59e5b" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 37433 } } peers { permanent_uuid: "693e6dd7718e4820b474b39d1149404b" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 35957 } } peers { permanent_uuid: "daa14490267b433f83f67f40a8d8367c" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 44447 } }
I20260811 19:04:10.642745 1751 leader_election.cc:288] T f67886eb1bdf4e15a23c7a2c19c3bd9b P daa14490267b433f83f67f40a8d8367c [CANDIDATE]: Term 1 election: Requested vote from peers 9302454b5cb3465b9872246e86b59e5b (127.13.109.3:37433), 693e6dd7718e4820b474b39d1149404b (127.13.109.1:35957)
I20260811 19:04:10.643123 1668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f67886eb1bdf4e15a23c7a2c19c3bd9b" candidate_uuid: "daa14490267b433f83f67f40a8d8367c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9302454b5cb3465b9872246e86b59e5b"
I20260811 19:04:10.643189 1368 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f67886eb1bdf4e15a23c7a2c19c3bd9b" candidate_uuid: "daa14490267b433f83f67f40a8d8367c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "693e6dd7718e4820b474b39d1149404b"
I20260811 19:04:10.643312 1668 raft_consensus.cc:3037] T f67886eb1bdf4e15a23c7a2c19c3bd9b P 9302454b5cb3465b9872246e86b59e5b [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:04:10.643327 1368 raft_consensus.cc:3037] T f67886eb1bdf4e15a23c7a2c19c3bd9b P 693e6dd7718e4820b474b39d1149404b [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:04:10.644431 1668 raft_consensus.cc:2454] T f67886eb1bdf4e15a23c7a2c19c3bd9b P 9302454b5cb3465b9872246e86b59e5b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate daa14490267b433f83f67f40a8d8367c in term 1.
I20260811 19:04:10.644441 1368 raft_consensus.cc:2454] T f67886eb1bdf4e15a23c7a2c19c3bd9b P 693e6dd7718e4820b474b39d1149404b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate daa14490267b433f83f67f40a8d8367c in term 1.
I20260811 19:04:10.644766 1444 leader_election.cc:302] T f67886eb1bdf4e15a23c7a2c19c3bd9b P daa14490267b433f83f67f40a8d8367c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9302454b5cb3465b9872246e86b59e5b, daa14490267b433f83f67f40a8d8367c; no voters:
I20260811 19:04:10.645015 1751 raft_consensus.cc:2781] T f67886eb1bdf4e15a23c7a2c19c3bd9b P daa14490267b433f83f67f40a8d8367c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:04:10.645275 1751 raft_consensus.cc:686] T f67886eb1bdf4e15a23c7a2c19c3bd9b P daa14490267b433f83f67f40a8d8367c [term 1 LEADER]: Becoming Leader. State: Replica: daa14490267b433f83f67f40a8d8367c, State: Running, Role: LEADER
I20260811 19:04:10.645676 1751 consensus_queue.cc:237] T f67886eb1bdf4e15a23c7a2c19c3bd9b P daa14490267b433f83f67f40a8d8367c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9302454b5cb3465b9872246e86b59e5b" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 37433 } } peers { permanent_uuid: "693e6dd7718e4820b474b39d1149404b" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 35957 } } peers { permanent_uuid: "daa14490267b433f83f67f40a8d8367c" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 44447 } }
I20260811 19:04:10.649219 1203 catalog_manager.cc:5495] T f67886eb1bdf4e15a23c7a2c19c3bd9b P daa14490267b433f83f67f40a8d8367c reported cstate change: term changed from 0 to 1, leader changed from <none> to daa14490267b433f83f67f40a8d8367c (127.13.109.2). New cstate: current_term: 1 leader_uuid: "daa14490267b433f83f67f40a8d8367c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9302454b5cb3465b9872246e86b59e5b" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 37433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "693e6dd7718e4820b474b39d1149404b" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 35957 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "daa14490267b433f83f67f40a8d8367c" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 44447 } health_report { overall_health: HEALTHY } } }
I20260811 19:04:10.672564 1740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:10.669093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40738 (local address 127.13.109.2:44447)
0811 19:04:10.669237 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:10.669242 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:10.669256 (+ 14us) server_negotiation.cc:407] Connection header received
0811 19:04:10.669331 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:10.669333 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:10.669392 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:10.669503 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:04:10.669748 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:10.670320 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:10.671135 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:10.671343 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:10.671419 (+ 76us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:10.671693 (+ 274us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:04:10.672103 (+ 410us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:04:10.672216 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:10.672407 (+ 191us) server_negotiation.cc:299] Negotiation successful
0811 19:04:10.672454 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 19:04:10.678735 1668 raft_consensus.cc:1261] T f67886eb1bdf4e15a23c7a2c19c3bd9b P 9302454b5cb3465b9872246e86b59e5b [term 1 FOLLOWER]: Refusing update from remote peer daa14490267b433f83f67f40a8d8367c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:04:10.678747 1368 raft_consensus.cc:1261] T f67886eb1bdf4e15a23c7a2c19c3bd9b P 693e6dd7718e4820b474b39d1149404b [term 1 FOLLOWER]: Refusing update from remote peer daa14490267b433f83f67f40a8d8367c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:04:10.679524 1751 consensus_queue.cc:1035] T f67886eb1bdf4e15a23c7a2c19c3bd9b P daa14490267b433f83f67f40a8d8367c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9302454b5cb3465b9872246e86b59e5b" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 37433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:04:10.679760 1753 consensus_queue.cc:1035] T f67886eb1bdf4e15a23c7a2c19c3bd9b P daa14490267b433f83f67f40a8d8367c [LEADER]: Connected to new peer: Peer: permanent_uuid: "693e6dd7718e4820b474b39d1149404b" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 35957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:04:10.688797 1759 mvcc.cc:205] Tried to move back new op lower bound from 7317401807571984384 to 7317401807447998464. Current Snapshot: MvccSnapshot[applied={T|T < 7317401807571984384}]
I20260811 19:04:10.698453 1203 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56420:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:04:10.698681 1203 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56420:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:04:10.701308 1203 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5da8dcdb2b1746b790ec6c8e29d037b5: Sending DeleteTablet for 3 replicas of tablet f67886eb1bdf4e15a23c7a2c19c3bd9b
I20260811 19:04:10.702278 1500 tablet_service.cc:1514] Processing DeleteTablet for tablet f67886eb1bdf4e15a23c7a2c19c3bd9b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:04:10 UTC) from {username='kudu', principal='kudu/127.13.109.62@KRBTEST.COM'} at 127.0.0.1:40722
I20260811 19:04:10.702512 1647 tablet_service.cc:1514] Processing DeleteTablet for tablet f67886eb1bdf4e15a23c7a2c19c3bd9b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:04:10 UTC) from {username='kudu', principal='kudu/127.13.109.62@KRBTEST.COM'} at 127.0.0.1:44324
I20260811 19:04:10.702476 1342 tablet_service.cc:1514] Processing DeleteTablet for tablet f67886eb1bdf4e15a23c7a2c19c3bd9b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:04:10 UTC) from {username='kudu', principal='kudu/127.13.109.62@KRBTEST.COM'} at 127.0.0.1:47684
I20260811 19:04:10.703086 1769 tablet_replica.cc:330] T f67886eb1bdf4e15a23c7a2c19c3bd9b P 9302454b5cb3465b9872246e86b59e5b: stopping tablet replica
I20260811 19:04:10.703107 1768 tablet_replica.cc:330] T f67886eb1bdf4e15a23c7a2c19c3bd9b P daa14490267b433f83f67f40a8d8367c: stopping tablet replica
I20260811 19:04:10.703510 1769 raft_consensus.cc:2229] T f67886eb1bdf4e15a23c7a2c19c3bd9b P 9302454b5cb3465b9872246e86b59e5b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:04:10.703497 1768 raft_consensus.cc:2229] T f67886eb1bdf4e15a23c7a2c19c3bd9b P daa14490267b433f83f67f40a8d8367c [term 1 LEADER]: Raft consensus shutting down.
I20260811 19:04:10.703975 1769 raft_consensus.cc:2258] T f67886eb1bdf4e15a23c7a2c19c3bd9b P 9302454b5cb3465b9872246e86b59e5b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:04:10.703974 1768 raft_consensus.cc:2258] T f67886eb1bdf4e15a23c7a2c19c3bd9b P daa14490267b433f83f67f40a8d8367c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:04:10.704041 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 1275
I20260811 19:04:10.706069 1769 ts_tablet_manager.cc:1905] T f67886eb1bdf4e15a23c7a2c19c3bd9b P 9302454b5cb3465b9872246e86b59e5b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:04:10.706984 1768 ts_tablet_manager.cc:1905] T f67886eb1bdf4e15a23c7a2c19c3bd9b P daa14490267b433f83f67f40a8d8367c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:04:10.709400 1769 ts_tablet_manager.cc:1918] T f67886eb1bdf4e15a23c7a2c19c3bd9b P 9302454b5cb3465b9872246e86b59e5b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:04:10.709846 1769 log.cc:1196] T f67886eb1bdf4e15a23c7a2c19c3bd9b P 9302454b5cb3465b9872246e86b59e5b: Deleting WAL directory at /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/minicluster-data/ts-2/wal/wals/f67886eb1bdf4e15a23c7a2c19c3bd9b
I20260811 19:04:10.710311 1769 ts_tablet_manager.cc:1939] T f67886eb1bdf4e15a23c7a2c19c3bd9b P 9302454b5cb3465b9872246e86b59e5b: Deleting consensus metadata
I20260811 19:04:10.710589 1768 ts_tablet_manager.cc:1918] T f67886eb1bdf4e15a23c7a2c19c3bd9b P daa14490267b433f83f67f40a8d8367c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:04:10.710723 1768 log.cc:1196] T f67886eb1bdf4e15a23c7a2c19c3bd9b P daa14490267b433f83f67f40a8d8367c: Deleting WAL directory at /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474792496492-13748-0/minicluster-data/ts-1/wal/wals/f67886eb1bdf4e15a23c7a2c19c3bd9b
I20260811 19:04:10.711119 1768 ts_tablet_manager.cc:1939] T f67886eb1bdf4e15a23c7a2c19c3bd9b P daa14490267b433f83f67f40a8d8367c: Deleting consensus metadata
I20260811 19:04:10.712019 1186 catalog_manager.cc:4841] TS 9302454b5cb3465b9872246e86b59e5b (127.13.109.3:37433): tablet f67886eb1bdf4e15a23c7a2c19c3bd9b (table test-table [id=692b4b47f0b24611902de399b2f4ac50]) successfully deleted
I20260811 19:04:10.712425 1187 catalog_manager.cc:4841] TS daa14490267b433f83f67f40a8d8367c (127.13.109.2:44447): tablet f67886eb1bdf4e15a23c7a2c19c3bd9b (table test-table [id=692b4b47f0b24611902de399b2f4ac50]) successfully deleted
W20260811 19:04:10.718313 1186 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.109.1:35957 (error 108)
I20260811 19:04:10.718417 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 1425
I20260811 19:04:10.720598 1741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:10.720133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.1:35957 (local address 127.0.0.1:47690)
0811 19:04:10.720371 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:10.720474 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.1:35957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128}
W20260811 19:04:10.721081 1186 catalog_manager.cc:4584] TS 693e6dd7718e4820b474b39d1149404b (127.13.109.1:35957): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f67886eb1bdf4e15a23c7a2c19c3bd9b: Network error: Client connection negotiation failed: client connection to 127.13.109.1:35957: connect: Connection refused (error 111)
I20260811 19:04:10.728299 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 1577
I20260811 19:04:10.737952 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 1168
2026-08-11T19:04:10Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2451 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-11T19:04:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:04:10Z Disabled control of system clock
I20260811 19:04:10.772749 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474792496492-13748-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474792496492-13748-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474792496492-13748-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.62:44259
--webserver_interface=127.13.109.62
--webserver_port=0
--builtin_ntp_servers=127.13.109.20:43055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.109.62:44259
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 19:04:10.897001 1775 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:04:10.897315 1775 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:04:10.897353 1775 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:04:10.901641 1775 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:04:10.901718 1775 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:04:10.901734 1775 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:04:10.901752 1775 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:04:10.901768 1775 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:04:10.907038 1775 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:43055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474792496492-13748-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474792496492-13748-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.109.62:44259
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.62:44259
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.109.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1775
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474792496492-13748-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:04:10.908334 1775 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:04:10.909890 1775 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:04:10.919210 1783 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:04:10.919224 1780 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:04:10.919255 1781 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:04:10.919436 1775 server_base.cc:1034] running on GCE node
I20260811 19:04:10.920324 1775 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:04:10.921542 1775 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:04:10.922752 1775 hybrid_clock.cc:648] HybridClock initialized: now 1786475050922724 us; error 41 us; skew 500 ppm
I20260811 19:04:10.924814 1775 webserver.cc:460] Webserver started at http://127.13.109.62:41181/ using document root <none> and password file <none>
I20260811 19:04:10.925460 1775 fs_manager.cc:359] Metadata directory not provided
I20260811 19:04:10.925566 1775 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:04:10.925887 1775 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:04:10.928102 1775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474792496492-13748-0/minicluster-data/master-0/data/instance:
uuid: "5ac9d771f2614670931496a09ae44b38"
format_stamp: "Formatted at 2026-08-11 19:04:10 on dist-test-slave-0kls"
server_key: "19c66dc2007ecdb4cd0a4d3d25e1b9ca"
server_key_iv: "b366042a1859760c142c12394c82c9ca"
server_key_version: "encryptionkey@0"
I20260811 19:04:10.928710 1775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474792496492-13748-0/minicluster-data/master-0/wal/instance:
uuid: "5ac9d771f2614670931496a09ae44b38"
format_stamp: "Formatted at 2026-08-11 19:04:10 on dist-test-slave-0kls"
server_key: "19c66dc2007ecdb4cd0a4d3d25e1b9ca"
server_key_iv: "b366042a1859760c142c12394c82c9ca"
server_key_version: "encryptionkey@0"
I20260811 19:04:10.932742 1775 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 19:04:10.935473 1790 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:04:10.936933 1775 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 19:04:10.937096 1775 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474792496492-13748-0/minicluster-data/master-0/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474792496492-13748-0/minicluster-data/master-0/wal
uuid: "5ac9d771f2614670931496a09ae44b38"
format_stamp: "Formatted at 2026-08-11 19:04:10 on dist-test-slave-0kls"
server_key: "19c66dc2007ecdb4cd0a4d3d25e1b9ca"
server_key_iv: "b366042a1859760c142c12394c82c9ca"
server_key_version: "encryptionkey@0"
I20260811 19:04:10.937233 1775 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474792496492-13748-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474792496492-13748-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474792496492-13748-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:04:10.963402 1775 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:04:10.964293 1775 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:04:10.964552 1775 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:04:10.972602 1775 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.62:44259
I20260811 19:04:10.972612 1848 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.62:44259 every 8 connection(s)
I20260811 19:04:10.973632 1775 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
I20260811 19:04:10.977005 1849 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:04:10.980214 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 1775
I20260811 19:04:10.980365 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474792496492-13748-0/minicluster-data/master-0/wal/instance
I20260811 19:04:10.980737 13748 external_mini_cluster.cc:1428] Setting key 33ec47e82a54e79ee72067170fcb93e0
I20260811 19:04:10.983997 1849 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ac9d771f2614670931496a09ae44b38: Bootstrap starting.
I20260811 19:04:10.986550 1849 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5ac9d771f2614670931496a09ae44b38: Neither blocks nor log segments found. Creating new log.
I20260811 19:04:10.987397 1849 log.cc:824] T 00000000000000000000000000000000 P 5ac9d771f2614670931496a09ae44b38: Log is configured to *not* fsync() on all Append() calls
I20260811 19:04:10.989650 1849 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ac9d771f2614670931496a09ae44b38: No bootstrap required, opened a new log
I20260811 19:04:10.991066 1852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:10.982401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39924 (local address 127.13.109.62:44259)
0811 19:04:10.983199 (+ 798us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:10.983224 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:10.983277 (+ 53us) server_negotiation.cc:407] Connection header received
0811 19:04:10.983976 (+ 699us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:10.983994 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:10.984501 (+ 507us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:10.984878 (+ 377us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:10.985370 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:10.986457 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:10.987471 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:10.988042 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:10.988231 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:10.989162 (+ 931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:04:10.989167 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:04:10.989182 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:04:10.989602 (+ 420us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:04:10.989666 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:04:10.989687 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:04:10.989851 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:10.990013 (+ 162us) server_negotiation.cc:299] Negotiation successful
0811 19:04:10.990304 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":168,"threads_started":1}
I20260811 19:04:10.994059 1849 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5ac9d771f2614670931496a09ae44b38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ac9d771f2614670931496a09ae44b38" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 44259 } }
I20260811 19:04:10.994333 1849 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5ac9d771f2614670931496a09ae44b38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:04:10.994427 1849 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5ac9d771f2614670931496a09ae44b38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ac9d771f2614670931496a09ae44b38, State: Initialized, Role: FOLLOWER
I20260811 19:04:10.994938 1849 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5ac9d771f2614670931496a09ae44b38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ac9d771f2614670931496a09ae44b38" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 44259 } }
I20260811 19:04:10.995116 1849 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5ac9d771f2614670931496a09ae44b38 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:04:10.995221 1849 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5ac9d771f2614670931496a09ae44b38 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:04:10.995326 1849 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5ac9d771f2614670931496a09ae44b38 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:04:10.996567 1849 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5ac9d771f2614670931496a09ae44b38 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ac9d771f2614670931496a09ae44b38" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 44259 } }
I20260811 19:04:10.996954 1849 leader_election.cc:302] T 00000000000000000000000000000000 P 5ac9d771f2614670931496a09ae44b38 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5ac9d771f2614670931496a09ae44b38; no voters:
I20260811 19:04:10.997257 1849 leader_election.cc:288] T 00000000000000000000000000000000 P 5ac9d771f2614670931496a09ae44b38 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:04:10.997394 1854 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5ac9d771f2614670931496a09ae44b38 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:04:10.997634 1854 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5ac9d771f2614670931496a09ae44b38 [term 1 LEADER]: Becoming Leader. State: Replica: 5ac9d771f2614670931496a09ae44b38, State: Running, Role: LEADER
I20260811 19:04:10.997979 1854 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5ac9d771f2614670931496a09ae44b38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ac9d771f2614670931496a09ae44b38" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 44259 } }
I20260811 19:04:10.998659 1849 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5ac9d771f2614670931496a09ae44b38 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:04:10.999845 1855 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ac9d771f2614670931496a09ae44b38 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5ac9d771f2614670931496a09ae44b38" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ac9d771f2614670931496a09ae44b38" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 44259 } } }
I20260811 19:04:10.999853 1856 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ac9d771f2614670931496a09ae44b38 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5ac9d771f2614670931496a09ae44b38. Latest consensus state: current_term: 1 leader_uuid: "5ac9d771f2614670931496a09ae44b38" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ac9d771f2614670931496a09ae44b38" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 44259 } } }
I20260811 19:04:11.000005 1855 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ac9d771f2614670931496a09ae44b38 [sys.catalog]: This master's current role is: LEADER
I20260811 19:04:11.000012 1856 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ac9d771f2614670931496a09ae44b38 [sys.catalog]: This master's current role is: LEADER
I20260811 19:04:11.000432 1863 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:04:11.003630 1863 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:04:11.009845 1863 catalog_manager.cc:1347] Generated new cluster ID: c6a0115959be43148aa605e1dcdb0ef2
I20260811 19:04:11.009948 1863 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:04:11.023727 1863 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:04:11.025115 1863 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:04:11.030685 1863 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5ac9d771f2614670931496a09ae44b38: Generated new TSK 0
I20260811 19:04:11.031569 1863 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 19:04:11.035336 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474792496492-13748-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474792496492-13748-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474792496492-13748-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.1:0
--local_ip_for_outbound_sockets=127.13.109.1
--webserver_interface=127.13.109.1
--webserver_port=0
--tserver_master_addrs=127.13.109.62:44259
--builtin_ntp_servers=127.13.109.20:43055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 19:04:11.156797 1876 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:04:11.157138 1876 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:04:11.157222 1876 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:04:11.161520 1876 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:04:11.161639 1876 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:04:11.161748 1876 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.1
I20260811 19:04:11.167116 1876 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:43055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474792496492-13748-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474792496492-13748-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.109.1
--webserver_port=0
--tserver_master_addrs=127.13.109.62:44259
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1876
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.1
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474792496492-13748-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:04:11.168442 1876 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:04:11.170006 1876 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:04:11.180083 1884 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:04:11.180174 1881 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:04:11.180167 1882 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:04:11.180451 1876 server_base.cc:1034] running on GCE node
I20260811 19:04:11.181005 1876 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:04:11.181694 1876 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:04:11.182895 1876 hybrid_clock.cc:648] HybridClock initialized: now 1786475051182878 us; error 25 us; skew 500 ppm
I20260811 19:04:11.184820 1876 webserver.cc:460] Webserver started at http://127.13.109.1:45197/ using document root <none> and password file <none>
I20260811 19:04:11.185472 1876 fs_manager.cc:359] Metadata directory not provided
I20260811 19:04:11.185580 1876 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:04:11.185910 1876 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:04:11.188167 1876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474792496492-13748-0/minicluster-data/ts-0/data/instance:
uuid: "ddd5a05b73cd4c46ab10a1588c743efb"
format_stamp: "Formatted at 2026-08-11 19:04:11 on dist-test-slave-0kls"
server_key: "7b7c1cc4d5580330e4bbd7e9de542afa"
server_key_iv: "82a9532ecece963af80c7c8a0571fd12"
server_key_version: "encryptionkey@0"
I20260811 19:04:11.188817 1876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474792496492-13748-0/minicluster-data/ts-0/wal/instance:
uuid: "ddd5a05b73cd4c46ab10a1588c743efb"
format_stamp: "Formatted at 2026-08-11 19:04:11 on dist-test-slave-0kls"
server_key: "7b7c1cc4d5580330e4bbd7e9de542afa"
server_key_iv: "82a9532ecece963af80c7c8a0571fd12"
server_key_version: "encryptionkey@0"
I20260811 19:04:11.192945 1876 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 19:04:11.195756 1890 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:04:11.197103 1876 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 19:04:11.197268 1876 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474792496492-13748-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474792496492-13748-0/minicluster-data/ts-0/wal
uuid: "ddd5a05b73cd4c46ab10a1588c743efb"
format_stamp: "Formatted at 2026-08-11 19:04:11 on dist-test-slave-0kls"
server_key: "7b7c1cc4d5580330e4bbd7e9de542afa"
server_key_iv: "82a9532ecece963af80c7c8a0571fd12"
server_key_version: "encryptionkey@0"
I20260811 19:04:11.197407 1876 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474792496492-13748-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474792496492-13748-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474792496492-13748-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:04:11.209893 1876 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:04:11.210633 1876 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:04:11.210857 1876 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:04:11.211504 1876 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:04:11.212517 1876 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:04:11.212595 1876 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:04:11.212666 1876 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:04:11.212725 1876 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:04:11.222430 1876 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.1:46367
I20260811 19:04:11.222458 2016 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.1:46367 every 8 connection(s)
I20260811 19:04:11.223527 1876 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
I20260811 19:04:11.233100 1852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:11.225544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.1:35679 (local address 127.13.109.62:44259)
0811 19:04:11.225702 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:11.225706 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:11.226869 (+ 1163us) server_negotiation.cc:407] Connection header received
0811 19:04:11.228287 (+ 1418us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:11.228290 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:11.228342 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:11.228426 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:11.229333 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:11.229954 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:11.231095 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:11.231327 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:11.231406 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:11.232555 (+ 1149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:04:11.232559 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:04:11.232561 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:04:11.232575 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:04:11.232594 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:04:11.232602 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:04:11.232672 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:11.232871 (+ 199us) server_negotiation.cc:299] Negotiation successful
0811 19:04:11.232939 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 19:04:11.233649 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 1876
I20260811 19:04:11.233843 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474792496492-13748-0/minicluster-data/ts-0/wal/instance
I20260811 19:04:11.234166 2019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:11.226022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:44259 (local address 127.13.109.1:35679)
0811 19:04:11.226697 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:11.226751 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 19:04:11.228059 (+ 1308us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:04:11.228607 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:04:11.228614 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:04:11.228697 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:04:11.229202 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:11.229215 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:11.230078 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:11.230081 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:11.230985 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:11.230996 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:11.231472 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:11.231474 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:11.231623 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:11.232267 (+ 644us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 19:04:11.232269 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:04:11.232311 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 19:04:11.232385 (+ 74us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:04:11.232687 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:04:11.232695 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:04:11.232699 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 19:04:11.232808 (+ 109us) client_negotiation.cc:240] Negotiation successful
0811 19:04:11.233051 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":117,"threads_started":1}
I20260811 19:04:11.234220 13748 external_mini_cluster.cc:1428] Setting key 515636eeff72291ace91fdc3f47e00d0
I20260811 19:04:11.235580 2017 heartbeater.cc:347] Connected to a master server at 127.13.109.62:44259
I20260811 19:04:11.235918 2017 heartbeater.cc:464] Registering TS with master...
I20260811 19:04:11.236644 2017 heartbeater.cc:511] Master 127.13.109.62:44259 requested a full tablet report, sending...
I20260811 19:04:11.236730 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474792496492-13748-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474792496492-13748-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474792496492-13748-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.2:0
--local_ip_for_outbound_sockets=127.13.109.2
--webserver_interface=127.13.109.2
--webserver_port=0
--tserver_master_addrs=127.13.109.62:44259
--builtin_ntp_servers=127.13.109.20:43055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 19:04:11.238432 1808 ts_manager.cc:194] Registered new tserver with Master: ddd5a05b73cd4c46ab10a1588c743efb (127.13.109.1:46367)
I20260811 19:04:11.239622 1808 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.109.1:35679
W20260811 19:04:11.356253 2020 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:04:11.356545 2020 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:04:11.356585 2020 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:04:11.360770 2020 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:04:11.360834 2020 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:04:11.360893 2020 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.2
I20260811 19:04:11.365922 2020 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:43055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474792496492-13748-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474792496492-13748-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.109.2
--webserver_port=0
--tserver_master_addrs=127.13.109.62:44259
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2020
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.2
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474792496492-13748-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:04:11.367106 2020 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:04:11.368585 2020 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:04:11.378572 2028 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:04:11.378655 2025 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:04:11.378578 2026 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:04:11.378643 2020 server_base.cc:1034] running on GCE node
I20260811 19:04:11.379248 2020 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:04:11.379916 2020 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:04:11.381114 2020 hybrid_clock.cc:648] HybridClock initialized: now 1786475051381066 us; error 62 us; skew 500 ppm
I20260811 19:04:11.383031 2020 webserver.cc:460] Webserver started at http://127.13.109.2:32785/ using document root <none> and password file <none>
I20260811 19:04:11.383623 2020 fs_manager.cc:359] Metadata directory not provided
I20260811 19:04:11.383731 2020 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:04:11.383976 2020 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:04:11.386237 2020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474792496492-13748-0/minicluster-data/ts-1/data/instance:
uuid: "418778429d0341bd82149a8ad23e17de"
format_stamp: "Formatted at 2026-08-11 19:04:11 on dist-test-slave-0kls"
server_key: "552906e3b21e69f528c2b90eb5257c21"
server_key_iv: "ea31acbe949fa31add3f41841e73c546"
server_key_version: "encryptionkey@0"
I20260811 19:04:11.386853 2020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474792496492-13748-0/minicluster-data/ts-1/wal/instance:
uuid: "418778429d0341bd82149a8ad23e17de"
format_stamp: "Formatted at 2026-08-11 19:04:11 on dist-test-slave-0kls"
server_key: "552906e3b21e69f528c2b90eb5257c21"
server_key_iv: "ea31acbe949fa31add3f41841e73c546"
server_key_version: "encryptionkey@0"
I20260811 19:04:11.391383 2020 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 19:04:11.393989 2035 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:04:11.395272 2020 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 19:04:11.395421 2020 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474792496492-13748-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474792496492-13748-0/minicluster-data/ts-1/wal
uuid: "418778429d0341bd82149a8ad23e17de"
format_stamp: "Formatted at 2026-08-11 19:04:11 on dist-test-slave-0kls"
server_key: "552906e3b21e69f528c2b90eb5257c21"
server_key_iv: "ea31acbe949fa31add3f41841e73c546"
server_key_version: "encryptionkey@0"
I20260811 19:04:11.395547 2020 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474792496492-13748-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474792496492-13748-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474792496492-13748-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:04:11.415838 2020 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:04:11.416594 2020 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:04:11.416821 2020 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:04:11.417474 2020 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:04:11.418552 2020 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:04:11.418629 2020 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:04:11.418749 2020 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:04:11.418805 2020 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:04:11.427732 2020 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.2:45569
I20260811 19:04:11.427760 2162 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.2:45569 every 8 connection(s)
I20260811 19:04:11.428648 2020 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
I20260811 19:04:11.435264 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 2020
I20260811 19:04:11.435422 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474792496492-13748-0/minicluster-data/ts-1/wal/instance
I20260811 19:04:11.435738 13748 external_mini_cluster.cc:1428] Setting key 7f032cc9983443df02e893249f0f560b
I20260811 19:04:11.437870 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474792496492-13748-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474792496492-13748-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474792496492-13748-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.3:0
--local_ip_for_outbound_sockets=127.13.109.3
--webserver_interface=127.13.109.3
--webserver_port=0
--tserver_master_addrs=127.13.109.62:44259
--builtin_ntp_servers=127.13.109.20:43055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 19:04:11.438222 1852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:11.430633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.2:42019 (local address 127.13.109.62:44259)
0811 19:04:11.430779 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:11.430783 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:11.431871 (+ 1088us) server_negotiation.cc:407] Connection header received
0811 19:04:11.433219 (+ 1348us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:11.433222 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:11.433277 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:11.433354 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:11.434292 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:11.434816 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:11.436143 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:11.436376 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:11.436501 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:11.437651 (+ 1150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:04:11.437656 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:04:11.437658 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:04:11.437678 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:04:11.437696 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:04:11.437706 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:04:11.437787 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:11.437999 (+ 212us) server_negotiation.cc:299] Negotiation successful
0811 19:04:11.438065 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 19:04:11.439150 2165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:11.431047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:44259 (local address 127.13.109.2:42019)
0811 19:04:11.431717 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:11.431759 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 19:04:11.433023 (+ 1264us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:04:11.433527 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:04:11.433534 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:04:11.433616 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:04:11.434147 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:11.434160 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:11.434946 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:11.434949 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:11.436009 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:11.436020 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:11.436547 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:11.436551 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:11.436727 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:11.437375 (+ 648us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 19:04:11.437377 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:04:11.437417 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 19:04:11.437485 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:04:11.437846 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:04:11.437855 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:04:11.437860 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 19:04:11.437998 (+ 138us) client_negotiation.cc:240] Negotiation successful
0811 19:04:11.438305 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":128,"threads_started":1}
I20260811 19:04:11.440500 2163 heartbeater.cc:347] Connected to a master server at 127.13.109.62:44259
I20260811 19:04:11.440762 2163 heartbeater.cc:464] Registering TS with master...
I20260811 19:04:11.441390 2163 heartbeater.cc:511] Master 127.13.109.62:44259 requested a full tablet report, sending...
I20260811 19:04:11.442684 1808 ts_manager.cc:194] Registered new tserver with Master: 418778429d0341bd82149a8ad23e17de (127.13.109.2:45569)
I20260811 19:04:11.443297 1808 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.109.2:42019
W20260811 19:04:11.562144 2166 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:04:11.562413 2166 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:04:11.562453 2166 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:04:11.566679 2166 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:04:11.566743 2166 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:04:11.566802 2166 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.3
I20260811 19:04:11.571741 2166 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:43055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474792496492-13748-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474792496492-13748-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.109.3
--webserver_port=0
--tserver_master_addrs=127.13.109.62:44259
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2166
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.3
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474792496492-13748-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:04:11.572953 2166 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:04:11.574438 2166 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:04:11.585250 2171 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:04:11.585408 2166 server_base.cc:1034] running on GCE node
W20260811 19:04:11.585250 2172 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:04:11.585330 2174 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:04:11.586179 2166 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:04:11.586977 2166 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:04:11.588188 2166 hybrid_clock.cc:648] HybridClock initialized: now 1786475051588162 us; error 37 us; skew 500 ppm
I20260811 19:04:11.590575 2166 webserver.cc:460] Webserver started at http://127.13.109.3:40831/ using document root <none> and password file <none>
I20260811 19:04:11.591243 2166 fs_manager.cc:359] Metadata directory not provided
I20260811 19:04:11.591348 2166 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:04:11.591643 2166 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:04:11.594113 2166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474792496492-13748-0/minicluster-data/ts-2/data/instance:
uuid: "e1926d06bfa14886b0ec998c9de854c7"
format_stamp: "Formatted at 2026-08-11 19:04:11 on dist-test-slave-0kls"
server_key: "c5315fddf84f408c69f57c319a7e492d"
server_key_iv: "63b25732e697a67bb5247f160586c344"
server_key_version: "encryptionkey@0"
I20260811 19:04:11.594769 2166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474792496492-13748-0/minicluster-data/ts-2/wal/instance:
uuid: "e1926d06bfa14886b0ec998c9de854c7"
format_stamp: "Formatted at 2026-08-11 19:04:11 on dist-test-slave-0kls"
server_key: "c5315fddf84f408c69f57c319a7e492d"
server_key_iv: "63b25732e697a67bb5247f160586c344"
server_key_version: "encryptionkey@0"
I20260811 19:04:11.599380 2166 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 19:04:11.602504 2180 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:04:11.604375 2166 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 19:04:11.604509 2166 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474792496492-13748-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474792496492-13748-0/minicluster-data/ts-2/wal
uuid: "e1926d06bfa14886b0ec998c9de854c7"
format_stamp: "Formatted at 2026-08-11 19:04:11 on dist-test-slave-0kls"
server_key: "c5315fddf84f408c69f57c319a7e492d"
server_key_iv: "63b25732e697a67bb5247f160586c344"
server_key_version: "encryptionkey@0"
I20260811 19:04:11.604653 2166 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474792496492-13748-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474792496492-13748-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474792496492-13748-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:04:11.627863 2166 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:04:11.628579 2166 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:04:11.628751 2166 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:04:11.629341 2166 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:04:11.630411 2166 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:04:11.630461 2166 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:04:11.630502 2166 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:04:11.630518 2166 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:04:11.639904 2166 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.3:41185
I20260811 19:04:11.639921 2308 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.3:41185 every 8 connection(s)
I20260811 19:04:11.640939 2166 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
I20260811 19:04:11.646939 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 2166
I20260811 19:04:11.647042 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474792496492-13748-0/minicluster-data/ts-2/wal/instance
I20260811 19:04:11.647347 13748 external_mini_cluster.cc:1428] Setting key ef1b75f7d2656aa643df561bb0546307
I20260811 19:04:11.651247 1852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:11.643018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.3:36021 (local address 127.13.109.62:44259)
0811 19:04:11.643195 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:11.643199 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:11.644242 (+ 1043us) server_negotiation.cc:407] Connection header received
0811 19:04:11.645711 (+ 1469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:11.645714 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:11.645806 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:11.645890 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:11.646834 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:11.647534 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:11.649048 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:11.649304 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:11.649433 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:11.650727 (+ 1294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:04:11.650730 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:04:11.650732 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:04:11.650745 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:04:11.650766 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:04:11.650775 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:04:11.650838 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:11.651050 (+ 212us) server_negotiation.cc:299] Negotiation successful
0811 19:04:11.651108 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 19:04:11.652240 2311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:11.643403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:44259 (local address 127.13.109.3:36021)
0811 19:04:11.644072 (+ 669us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:11.644114 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 19:04:11.645479 (+ 1365us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:04:11.646070 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:04:11.646078 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:04:11.646164 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:04:11.646675 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:11.646687 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:11.647687 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:11.647691 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:11.648922 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:11.648931 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:11.649536 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:11.649543 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:11.649745 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:11.650444 (+ 699us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 19:04:11.650446 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:04:11.650491 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 19:04:11.650567 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:04:11.650890 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:04:11.650899 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:04:11.650904 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 19:04:11.651029 (+ 125us) client_negotiation.cc:240] Negotiation successful
0811 19:04:11.651282 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":116,"threads_started":1}
I20260811 19:04:11.653502 2309 heartbeater.cc:347] Connected to a master server at 127.13.109.62:44259
I20260811 19:04:11.653740 2309 heartbeater.cc:464] Registering TS with master...
I20260811 19:04:11.654366 2309 heartbeater.cc:511] Master 127.13.109.62:44259 requested a full tablet report, sending...
I20260811 19:04:11.655485 1808 ts_manager.cc:194] Registered new tserver with Master: e1926d06bfa14886b0ec998c9de854c7 (127.13.109.3:41185)
I20260811 19:04:11.656071 1808 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.109.3:36021
I20260811 19:04:11.661736 13748 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 19:04:11.668359 1852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:11.663953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39940 (local address 127.13.109.62:44259)
0811 19:04:11.664166 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:11.664172 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:11.664336 (+ 164us) server_negotiation.cc:407] Connection header received
0811 19:04:11.664519 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:11.664522 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:11.664578 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:11.664671 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:11.664992 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:11.665546 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:11.666507 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:11.666744 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:11.666862 (+ 118us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:11.667823 (+ 961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:04:11.667827 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:04:11.667830 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:04:11.667843 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:04:11.667872 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:04:11.667881 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:04:11.668020 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:11.668155 (+ 135us) server_negotiation.cc:299] Negotiation successful
0811 19:04:11.668198 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114}
I20260811 19:04:11.671561 1808 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39940:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 19:04:11.674949 1808 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 19:04:11.690456 2326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:11.685409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52936 (local address 127.13.109.3:41185)
0811 19:04:11.685860 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:11.685872 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:11.685897 (+ 25us) server_negotiation.cc:407] Connection header received
0811 19:04:11.685972 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:11.685976 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:11.686164 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:11.686323 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:04:11.686585 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:11.687679 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:11.689071 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:11.689682 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:11.689836 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:11.690058 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:11.690167 (+ 109us) server_negotiation.cc:299] Negotiation successful
0811 19:04:11.690275 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":106,"threads_started":1}
I20260811 19:04:11.690451 2323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:11.684424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.1:46367 (local address 127.0.0.1:34728)
0811 19:04:11.684997 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:11.685011 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 19:04:11.685093 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:04:11.686169 (+ 1076us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:04:11.686172 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:04:11.686189 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:04:11.686282 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:11.686290 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:11.687349 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:11.687356 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:11.688771 (+ 1415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:11.688783 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:11.689981 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:11.689985 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:11.690104 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:11.690133 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 19:04:11.690181 (+ 48us) client_negotiation.cc:240] Negotiation successful
0811 19:04:11.690285 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"mutex_wait_us":86,"thread_start_us":121,"threads_started":1}
I20260811 19:04:11.690691 2320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:11.684174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.2:45569 (local address 127.0.0.1:41558)
0811 19:04:11.684563 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:11.684618 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 19:04:11.684746 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:04:11.685706 (+ 960us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:04:11.685713 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:04:11.685752 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:04:11.685974 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:11.685982 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:11.687706 (+ 1724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:11.687710 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:11.689061 (+ 1351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:11.689070 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:11.690126 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:11.690130 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:11.690245 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:11.690267 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 19:04:11.690309 (+ 42us) client_negotiation.cc:240] Negotiation successful
0811 19:04:11.690378 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"spinlock_wait_cycles":4608,"thread_start_us":87,"threads_started":1}
I20260811 19:04:11.690987 2324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:11.684935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.3:41185 (local address 127.0.0.1:52936)
0811 19:04:11.685707 (+ 772us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:11.685722 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 19:04:11.685846 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:04:11.686346 (+ 500us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:04:11.686352 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:04:11.686373 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:04:11.686446 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:11.686456 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:11.687823 (+ 1367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:11.687827 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:11.688933 (+ 1106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:11.688944 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:11.689841 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:11.689844 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:11.689962 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:11.690003 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 19:04:11.690131 (+ 128us) client_negotiation.cc:240] Negotiation successful
0811 19:04:11.690892 (+ 761us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":691,"mutex_wait_us":4,"spinlock_wait_cycles":1408,"thread_start_us":98,"threads_started":1}
I20260811 19:04:11.691053 2322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:11.684376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41558 (local address 127.13.109.2:45569)
0811 19:04:11.685147 (+ 771us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:11.685162 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:11.685190 (+ 28us) server_negotiation.cc:407] Connection header received
0811 19:04:11.685273 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:11.685280 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:11.685512 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:11.686142 (+ 630us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:04:11.686311 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:11.687566 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:11.689201 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:11.690003 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:11.690534 (+ 531us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:11.690736 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:11.690789 (+ 53us) server_negotiation.cc:299] Negotiation successful
0811 19:04:11.690892 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":654,"thread_start_us":151,"threads_started":1}
I20260811 19:04:11.691710 2325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:11.685060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34728 (local address 127.13.109.1:46367)
0811 19:04:11.685516 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:11.685698 (+ 182us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:11.685732 (+ 34us) server_negotiation.cc:407] Connection header received
0811 19:04:11.685842 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:11.685848 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:11.686029 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:11.686201 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:04:11.686415 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:11.687226 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:11.688918 (+ 1692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:11.689828 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:11.690276 (+ 448us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:11.690473 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:11.691411 (+ 938us) server_negotiation.cc:299] Negotiation successful
0811 19:04:11.691544 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":135,"threads_started":1}
I20260811 19:04:11.694352 2238 tablet_service.cc:1467] Processing CreateTablet for tablet 5f6e066292cd440e9e3abb8d40047ada (DEFAULT_TABLE table=test-table [id=92b66a1c99914dbf8064a40cbc3c87fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:04:11.694383 2088 tablet_service.cc:1467] Processing CreateTablet for tablet 5f6e066292cd440e9e3abb8d40047ada (DEFAULT_TABLE table=test-table [id=92b66a1c99914dbf8064a40cbc3c87fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:04:11.694383 1946 tablet_service.cc:1467] Processing CreateTablet for tablet 5f6e066292cd440e9e3abb8d40047ada (DEFAULT_TABLE table=test-table [id=92b66a1c99914dbf8064a40cbc3c87fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:04:11.695505 2238 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f6e066292cd440e9e3abb8d40047ada. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:04:11.695508 1946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f6e066292cd440e9e3abb8d40047ada. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:04:11.695508 2088 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f6e066292cd440e9e3abb8d40047ada. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:04:11.703325 2330 tablet_bootstrap.cc:492] T 5f6e066292cd440e9e3abb8d40047ada P e1926d06bfa14886b0ec998c9de854c7: Bootstrap starting.
I20260811 19:04:11.704113 2329 tablet_bootstrap.cc:492] T 5f6e066292cd440e9e3abb8d40047ada P 418778429d0341bd82149a8ad23e17de: Bootstrap starting.
I20260811 19:04:11.704737 2328 tablet_bootstrap.cc:492] T 5f6e066292cd440e9e3abb8d40047ada P ddd5a05b73cd4c46ab10a1588c743efb: Bootstrap starting.
I20260811 19:04:11.706058 2330 tablet_bootstrap.cc:654] T 5f6e066292cd440e9e3abb8d40047ada P e1926d06bfa14886b0ec998c9de854c7: Neither blocks nor log segments found. Creating new log.
I20260811 19:04:11.706091 2329 tablet_bootstrap.cc:654] T 5f6e066292cd440e9e3abb8d40047ada P 418778429d0341bd82149a8ad23e17de: Neither blocks nor log segments found. Creating new log.
I20260811 19:04:11.706879 2328 tablet_bootstrap.cc:654] T 5f6e066292cd440e9e3abb8d40047ada P ddd5a05b73cd4c46ab10a1588c743efb: Neither blocks nor log segments found. Creating new log.
I20260811 19:04:11.706990 2330 log.cc:824] T 5f6e066292cd440e9e3abb8d40047ada P e1926d06bfa14886b0ec998c9de854c7: Log is configured to *not* fsync() on all Append() calls
I20260811 19:04:11.706991 2329 log.cc:824] T 5f6e066292cd440e9e3abb8d40047ada P 418778429d0341bd82149a8ad23e17de: Log is configured to *not* fsync() on all Append() calls
I20260811 19:04:11.707712 2328 log.cc:824] T 5f6e066292cd440e9e3abb8d40047ada P ddd5a05b73cd4c46ab10a1588c743efb: Log is configured to *not* fsync() on all Append() calls
I20260811 19:04:11.709434 2329 tablet_bootstrap.cc:492] T 5f6e066292cd440e9e3abb8d40047ada P 418778429d0341bd82149a8ad23e17de: No bootstrap required, opened a new log
I20260811 19:04:11.709673 2329 ts_tablet_manager.cc:1397] T 5f6e066292cd440e9e3abb8d40047ada P 418778429d0341bd82149a8ad23e17de: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 19:04:11.709692 2328 tablet_bootstrap.cc:492] T 5f6e066292cd440e9e3abb8d40047ada P ddd5a05b73cd4c46ab10a1588c743efb: No bootstrap required, opened a new log
I20260811 19:04:11.709946 2328 ts_tablet_manager.cc:1397] T 5f6e066292cd440e9e3abb8d40047ada P ddd5a05b73cd4c46ab10a1588c743efb: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260811 19:04:11.711231 2330 tablet_bootstrap.cc:492] T 5f6e066292cd440e9e3abb8d40047ada P e1926d06bfa14886b0ec998c9de854c7: No bootstrap required, opened a new log
I20260811 19:04:11.711431 2330 ts_tablet_manager.cc:1397] T 5f6e066292cd440e9e3abb8d40047ada P e1926d06bfa14886b0ec998c9de854c7: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260811 19:04:11.713372 2328 raft_consensus.cc:348] T 5f6e066292cd440e9e3abb8d40047ada P ddd5a05b73cd4c46ab10a1588c743efb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "418778429d0341bd82149a8ad23e17de" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 45569 } } peers { permanent_uuid: "e1926d06bfa14886b0ec998c9de854c7" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 41185 } } peers { permanent_uuid: "ddd5a05b73cd4c46ab10a1588c743efb" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 46367 } }
I20260811 19:04:11.713655 2328 raft_consensus.cc:374] T 5f6e066292cd440e9e3abb8d40047ada P ddd5a05b73cd4c46ab10a1588c743efb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:04:11.713712 2328 raft_consensus.cc:729] T 5f6e066292cd440e9e3abb8d40047ada P ddd5a05b73cd4c46ab10a1588c743efb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ddd5a05b73cd4c46ab10a1588c743efb, State: Initialized, Role: FOLLOWER
I20260811 19:04:11.714460 2328 consensus_queue.cc:260] T 5f6e066292cd440e9e3abb8d40047ada P ddd5a05b73cd4c46ab10a1588c743efb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "418778429d0341bd82149a8ad23e17de" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 45569 } } peers { permanent_uuid: "e1926d06bfa14886b0ec998c9de854c7" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 41185 } } peers { permanent_uuid: "ddd5a05b73cd4c46ab10a1588c743efb" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 46367 } }
I20260811 19:04:11.715207 2017 heartbeater.cc:503] Master 127.13.109.62:44259 was elected leader, sending a full tablet report...
I20260811 19:04:11.715482 2329 raft_consensus.cc:348] T 5f6e066292cd440e9e3abb8d40047ada P 418778429d0341bd82149a8ad23e17de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "418778429d0341bd82149a8ad23e17de" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 45569 } } peers { permanent_uuid: "e1926d06bfa14886b0ec998c9de854c7" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 41185 } } peers { permanent_uuid: "ddd5a05b73cd4c46ab10a1588c743efb" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 46367 } }
I20260811 19:04:11.715857 2329 raft_consensus.cc:374] T 5f6e066292cd440e9e3abb8d40047ada P 418778429d0341bd82149a8ad23e17de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:04:11.715875 2328 ts_tablet_manager.cc:1428] T 5f6e066292cd440e9e3abb8d40047ada P ddd5a05b73cd4c46ab10a1588c743efb: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260811 19:04:11.715938 2329 raft_consensus.cc:729] T 5f6e066292cd440e9e3abb8d40047ada P 418778429d0341bd82149a8ad23e17de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 418778429d0341bd82149a8ad23e17de, State: Initialized, Role: FOLLOWER
I20260811 19:04:11.716012 2330 raft_consensus.cc:348] T 5f6e066292cd440e9e3abb8d40047ada P e1926d06bfa14886b0ec998c9de854c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "418778429d0341bd82149a8ad23e17de" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 45569 } } peers { permanent_uuid: "e1926d06bfa14886b0ec998c9de854c7" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 41185 } } peers { permanent_uuid: "ddd5a05b73cd4c46ab10a1588c743efb" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 46367 } }
I20260811 19:04:11.716250 2330 raft_consensus.cc:374] T 5f6e066292cd440e9e3abb8d40047ada P e1926d06bfa14886b0ec998c9de854c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:04:11.716298 2330 raft_consensus.cc:729] T 5f6e066292cd440e9e3abb8d40047ada P e1926d06bfa14886b0ec998c9de854c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1926d06bfa14886b0ec998c9de854c7, State: Initialized, Role: FOLLOWER
I20260811 19:04:11.716455 2329 consensus_queue.cc:260] T 5f6e066292cd440e9e3abb8d40047ada P 418778429d0341bd82149a8ad23e17de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "418778429d0341bd82149a8ad23e17de" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 45569 } } peers { permanent_uuid: "e1926d06bfa14886b0ec998c9de854c7" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 41185 } } peers { permanent_uuid: "ddd5a05b73cd4c46ab10a1588c743efb" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 46367 } }
I20260811 19:04:11.716751 2330 consensus_queue.cc:260] T 5f6e066292cd440e9e3abb8d40047ada P e1926d06bfa14886b0ec998c9de854c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "418778429d0341bd82149a8ad23e17de" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 45569 } } peers { permanent_uuid: "e1926d06bfa14886b0ec998c9de854c7" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 41185 } } peers { permanent_uuid: "ddd5a05b73cd4c46ab10a1588c743efb" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 46367 } }
I20260811 19:04:11.717199 2163 heartbeater.cc:503] Master 127.13.109.62:44259 was elected leader, sending a full tablet report...
I20260811 19:04:11.717520 2329 ts_tablet_manager.cc:1428] T 5f6e066292cd440e9e3abb8d40047ada P 418778429d0341bd82149a8ad23e17de: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260811 19:04:11.717562 2330 ts_tablet_manager.cc:1428] T 5f6e066292cd440e9e3abb8d40047ada P e1926d06bfa14886b0ec998c9de854c7: Time spent starting tablet: real 0.006s user 0.001s sys 0.006s
I20260811 19:04:11.717907 2309 heartbeater.cc:503] Master 127.13.109.62:44259 was elected leader, sending a full tablet report...
W20260811 19:04:11.724822 2018 tablet.cc:2369] T 5f6e066292cd440e9e3abb8d40047ada P ddd5a05b73cd4c46ab10a1588c743efb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 19:04:11.817423 2336 raft_consensus.cc:482] T 5f6e066292cd440e9e3abb8d40047ada P e1926d06bfa14886b0ec998c9de854c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:04:11.817710 2336 raft_consensus.cc:504] T 5f6e066292cd440e9e3abb8d40047ada P e1926d06bfa14886b0ec998c9de854c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "418778429d0341bd82149a8ad23e17de" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 45569 } } peers { permanent_uuid: "e1926d06bfa14886b0ec998c9de854c7" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 41185 } } peers { permanent_uuid: "ddd5a05b73cd4c46ab10a1588c743efb" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 46367 } }
I20260811 19:04:11.819016 2336 leader_election.cc:288] T 5f6e066292cd440e9e3abb8d40047ada P e1926d06bfa14886b0ec998c9de854c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 418778429d0341bd82149a8ad23e17de (127.13.109.2:45569), ddd5a05b73cd4c46ab10a1588c743efb (127.13.109.1:46367)
I20260811 19:04:11.823261 2322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:11.819423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.3:49757 (local address 127.13.109.2:45569)
0811 19:04:11.819582 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:11.819587 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:11.819935 (+ 348us) server_negotiation.cc:407] Connection header received
0811 19:04:11.820061 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:11.820064 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:11.820118 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:11.820197 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:04:11.820515 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:11.821039 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:11.822242 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:11.822672 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:11.822814 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:11.822868 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:11.823053 (+ 185us) server_negotiation.cc:299] Negotiation successful
0811 19:04:11.823130 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 19:04:11.823276 2337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:11.819361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.2:45569 (local address 127.13.109.3:49757)
0811 19:04:11.819826 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:11.819840 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 19:04:11.819943 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:04:11.820231 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:04:11.820233 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:04:11.820248 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:04:11.820392 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:11.820399 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:11.821163 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:11.821166 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:11.822068 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:11.822078 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:11.822814 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:11.822818 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:11.822922 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:11.822942 (+ 20us) client_negotiation.cc:769] Sending connection context
0811 19:04:11.823036 (+ 94us) client_negotiation.cc:240] Negotiation successful
0811 19:04:11.823115 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":361,"mutex_wait_us":3,"thread_start_us":130,"threads_started":1}
I20260811 19:04:11.823378 2325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:11.819449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.3:45911 (local address 127.13.109.1:46367)
0811 19:04:11.819580 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:11.819585 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:11.819694 (+ 109us) server_negotiation.cc:407] Connection header received
0811 19:04:11.819881 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:11.819888 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:11.819954 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:11.820034 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:04:11.820395 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:11.820949 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:11.822307 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:11.822725 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:11.822886 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:11.822938 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:11.823140 (+ 202us) server_negotiation.cc:299] Negotiation successful
0811 19:04:11.823217 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 19:04:11.823406 2311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:11.819334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.1:46367 (local address 127.13.109.3:45911)
0811 19:04:11.819582 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:11.819604 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 19:04:11.819722 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:04:11.820071 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:04:11.820073 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:04:11.820092 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:04:11.820253 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:11.820263 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:11.821078 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:11.821085 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:11.822161 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:11.822175 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:11.822886 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:11.822889 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:11.822997 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:11.823021 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 19:04:11.823120 (+ 99us) client_negotiation.cc:240] Negotiation successful
0811 19:04:11.823195 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"mutex_wait_us":77}
I20260811 19:04:11.823983 2111 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f6e066292cd440e9e3abb8d40047ada" candidate_uuid: "e1926d06bfa14886b0ec998c9de854c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "418778429d0341bd82149a8ad23e17de" is_pre_election: true
I20260811 19:04:11.824028 1967 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f6e066292cd440e9e3abb8d40047ada" candidate_uuid: "e1926d06bfa14886b0ec998c9de854c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ddd5a05b73cd4c46ab10a1588c743efb" is_pre_election: true
I20260811 19:04:11.824328 2111 raft_consensus.cc:2454] T 5f6e066292cd440e9e3abb8d40047ada P 418778429d0341bd82149a8ad23e17de [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1926d06bfa14886b0ec998c9de854c7 in term 0.
I20260811 19:04:11.824354 1967 raft_consensus.cc:2454] T 5f6e066292cd440e9e3abb8d40047ada P ddd5a05b73cd4c46ab10a1588c743efb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1926d06bfa14886b0ec998c9de854c7 in term 0.
I20260811 19:04:11.824868 2184 leader_election.cc:302] T 5f6e066292cd440e9e3abb8d40047ada P e1926d06bfa14886b0ec998c9de854c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 418778429d0341bd82149a8ad23e17de, e1926d06bfa14886b0ec998c9de854c7; no voters:
I20260811 19:04:11.825189 2336 raft_consensus.cc:2781] T 5f6e066292cd440e9e3abb8d40047ada P e1926d06bfa14886b0ec998c9de854c7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 19:04:11.825270 2336 raft_consensus.cc:482] T 5f6e066292cd440e9e3abb8d40047ada P e1926d06bfa14886b0ec998c9de854c7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 19:04:11.825301 2336 raft_consensus.cc:3037] T 5f6e066292cd440e9e3abb8d40047ada P e1926d06bfa14886b0ec998c9de854c7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:04:11.826421 2336 raft_consensus.cc:504] T 5f6e066292cd440e9e3abb8d40047ada P e1926d06bfa14886b0ec998c9de854c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "418778429d0341bd82149a8ad23e17de" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 45569 } } peers { permanent_uuid: "e1926d06bfa14886b0ec998c9de854c7" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 41185 } } peers { permanent_uuid: "ddd5a05b73cd4c46ab10a1588c743efb" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 46367 } }
I20260811 19:04:11.827023 2336 leader_election.cc:288] T 5f6e066292cd440e9e3abb8d40047ada P e1926d06bfa14886b0ec998c9de854c7 [CANDIDATE]: Term 1 election: Requested vote from peers 418778429d0341bd82149a8ad23e17de (127.13.109.2:45569), ddd5a05b73cd4c46ab10a1588c743efb (127.13.109.1:46367)
I20260811 19:04:11.827425 2111 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f6e066292cd440e9e3abb8d40047ada" candidate_uuid: "e1926d06bfa14886b0ec998c9de854c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "418778429d0341bd82149a8ad23e17de"
I20260811 19:04:11.827481 1967 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f6e066292cd440e9e3abb8d40047ada" candidate_uuid: "e1926d06bfa14886b0ec998c9de854c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ddd5a05b73cd4c46ab10a1588c743efb"
I20260811 19:04:11.827601 2111 raft_consensus.cc:3037] T 5f6e066292cd440e9e3abb8d40047ada P 418778429d0341bd82149a8ad23e17de [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:04:11.827651 1967 raft_consensus.cc:3037] T 5f6e066292cd440e9e3abb8d40047ada P ddd5a05b73cd4c46ab10a1588c743efb [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:04:11.828773 1967 raft_consensus.cc:2454] T 5f6e066292cd440e9e3abb8d40047ada P ddd5a05b73cd4c46ab10a1588c743efb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1926d06bfa14886b0ec998c9de854c7 in term 1.
I20260811 19:04:11.828792 2111 raft_consensus.cc:2454] T 5f6e066292cd440e9e3abb8d40047ada P 418778429d0341bd82149a8ad23e17de [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1926d06bfa14886b0ec998c9de854c7 in term 1.
I20260811 19:04:11.829223 2181 leader_election.cc:302] T 5f6e066292cd440e9e3abb8d40047ada P e1926d06bfa14886b0ec998c9de854c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 418778429d0341bd82149a8ad23e17de, ddd5a05b73cd4c46ab10a1588c743efb, e1926d06bfa14886b0ec998c9de854c7; no voters:
I20260811 19:04:11.829437 2336 raft_consensus.cc:2781] T 5f6e066292cd440e9e3abb8d40047ada P e1926d06bfa14886b0ec998c9de854c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:04:11.829696 2336 raft_consensus.cc:686] T 5f6e066292cd440e9e3abb8d40047ada P e1926d06bfa14886b0ec998c9de854c7 [term 1 LEADER]: Becoming Leader. State: Replica: e1926d06bfa14886b0ec998c9de854c7, State: Running, Role: LEADER
I20260811 19:04:11.830156 2336 consensus_queue.cc:237] T 5f6e066292cd440e9e3abb8d40047ada P e1926d06bfa14886b0ec998c9de854c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "418778429d0341bd82149a8ad23e17de" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 45569 } } peers { permanent_uuid: "e1926d06bfa14886b0ec998c9de854c7" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 41185 } } peers { permanent_uuid: "ddd5a05b73cd4c46ab10a1588c743efb" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 46367 } }
I20260811 19:04:11.833664 1808 catalog_manager.cc:5495] T 5f6e066292cd440e9e3abb8d40047ada P e1926d06bfa14886b0ec998c9de854c7 reported cstate change: term changed from 0 to 1, leader changed from <none> to e1926d06bfa14886b0ec998c9de854c7 (127.13.109.3). New cstate: current_term: 1 leader_uuid: "e1926d06bfa14886b0ec998c9de854c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "418778429d0341bd82149a8ad23e17de" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 45569 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e1926d06bfa14886b0ec998c9de854c7" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 41185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ddd5a05b73cd4c46ab10a1588c743efb" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 46367 } health_report { overall_health: UNKNOWN } } }
I20260811 19:04:11.846943 2326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:11.843269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52952 (local address 127.13.109.3:41185)
0811 19:04:11.843405 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:11.843409 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:11.843422 (+ 13us) server_negotiation.cc:407] Connection header received
0811 19:04:11.843495 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:11.843497 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:11.843570 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:11.843664 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:04:11.843908 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:11.844405 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:11.845283 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:11.845505 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:11.845600 (+ 95us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:11.845976 (+ 376us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:04:11.846424 (+ 448us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:04:11.846548 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:11.846748 (+ 200us) server_negotiation.cc:299] Negotiation successful
0811 19:04:11.846800 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 19:04:11.853500 2111 raft_consensus.cc:1261] T 5f6e066292cd440e9e3abb8d40047ada P 418778429d0341bd82149a8ad23e17de [term 1 FOLLOWER]: Refusing update from remote peer e1926d06bfa14886b0ec998c9de854c7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:04:11.853502 1967 raft_consensus.cc:1261] T 5f6e066292cd440e9e3abb8d40047ada P ddd5a05b73cd4c46ab10a1588c743efb [term 1 FOLLOWER]: Refusing update from remote peer e1926d06bfa14886b0ec998c9de854c7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:04:11.854419 2338 consensus_queue.cc:1035] T 5f6e066292cd440e9e3abb8d40047ada P e1926d06bfa14886b0ec998c9de854c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ddd5a05b73cd4c46ab10a1588c743efb" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 46367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:04:11.854638 2336 consensus_queue.cc:1035] T 5f6e066292cd440e9e3abb8d40047ada P e1926d06bfa14886b0ec998c9de854c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "418778429d0341bd82149a8ad23e17de" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 45569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:04:11.864993 2343 mvcc.cc:205] Tried to move back new op lower bound from 7317401812383350784 to 7317401812299272192. Current Snapshot: MvccSnapshot[applied={T|T < 7317401812383350784}]
I20260811 19:04:11.872402 1808 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39940:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:04:11.872577 1808 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39940:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:04:11.875376 1808 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5ac9d771f2614670931496a09ae44b38: Sending DeleteTablet for 3 replicas of tablet 5f6e066292cd440e9e3abb8d40047ada
I20260811 19:04:11.876420 2088 tablet_service.cc:1514] Processing DeleteTablet for tablet 5f6e066292cd440e9e3abb8d40047ada with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:04:11 UTC) from {username='slave'} at 127.0.0.1:41558
I20260811 19:04:11.876420 1946 tablet_service.cc:1514] Processing DeleteTablet for tablet 5f6e066292cd440e9e3abb8d40047ada with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:04:11 UTC) from {username='slave'} at 127.0.0.1:34728
I20260811 19:04:11.877048 2351 tablet_replica.cc:330] T 5f6e066292cd440e9e3abb8d40047ada P 418778429d0341bd82149a8ad23e17de: stopping tablet replica
I20260811 19:04:11.877187 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 1876
I20260811 19:04:11.877183 2238 tablet_service.cc:1514] Processing DeleteTablet for tablet 5f6e066292cd440e9e3abb8d40047ada with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:04:11 UTC) from {username='slave'} at 127.0.0.1:52936
I20260811 19:04:11.880620 2351 raft_consensus.cc:2229] T 5f6e066292cd440e9e3abb8d40047ada P 418778429d0341bd82149a8ad23e17de [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:04:11.880951 2354 tablet_replica.cc:330] T 5f6e066292cd440e9e3abb8d40047ada P e1926d06bfa14886b0ec998c9de854c7: stopping tablet replica
I20260811 19:04:11.881119 2351 raft_consensus.cc:2258] T 5f6e066292cd440e9e3abb8d40047ada P 418778429d0341bd82149a8ad23e17de [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:04:11.881299 2354 raft_consensus.cc:2229] T 5f6e066292cd440e9e3abb8d40047ada P e1926d06bfa14886b0ec998c9de854c7 [term 1 LEADER]: Raft consensus shutting down.
I20260811 19:04:11.881858 2354 raft_consensus.cc:2258] T 5f6e066292cd440e9e3abb8d40047ada P e1926d06bfa14886b0ec998c9de854c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:04:11.883170 2351 ts_tablet_manager.cc:1905] T 5f6e066292cd440e9e3abb8d40047ada P 418778429d0341bd82149a8ad23e17de: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:04:11.883249 2354 ts_tablet_manager.cc:1905] T 5f6e066292cd440e9e3abb8d40047ada P e1926d06bfa14886b0ec998c9de854c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:04:11.886234 2354 ts_tablet_manager.cc:1918] T 5f6e066292cd440e9e3abb8d40047ada P e1926d06bfa14886b0ec998c9de854c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:04:11.886350 2354 log.cc:1196] T 5f6e066292cd440e9e3abb8d40047ada P e1926d06bfa14886b0ec998c9de854c7: Deleting WAL directory at /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474792496492-13748-0/minicluster-data/ts-2/wal/wals/5f6e066292cd440e9e3abb8d40047ada
I20260811 19:04:11.886683 2354 ts_tablet_manager.cc:1939] T 5f6e066292cd440e9e3abb8d40047ada P e1926d06bfa14886b0ec998c9de854c7: Deleting consensus metadata
I20260811 19:04:11.886900 2351 ts_tablet_manager.cc:1918] T 5f6e066292cd440e9e3abb8d40047ada P 418778429d0341bd82149a8ad23e17de: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:04:11.887019 2351 log.cc:1196] T 5f6e066292cd440e9e3abb8d40047ada P 418778429d0341bd82149a8ad23e17de: Deleting WAL directory at /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474792496492-13748-0/minicluster-data/ts-1/wal/wals/5f6e066292cd440e9e3abb8d40047ada
I20260811 19:04:11.887332 2351 ts_tablet_manager.cc:1939] T 5f6e066292cd440e9e3abb8d40047ada P 418778429d0341bd82149a8ad23e17de: Deleting consensus metadata
I20260811 19:04:11.888638 1794 catalog_manager.cc:4841] TS e1926d06bfa14886b0ec998c9de854c7 (127.13.109.3:41185): tablet 5f6e066292cd440e9e3abb8d40047ada (table test-table [id=92b66a1c99914dbf8064a40cbc3c87fa]) successfully deleted
I20260811 19:04:11.888898 1794 catalog_manager.cc:4841] TS 418778429d0341bd82149a8ad23e17de (127.13.109.2:45569): tablet 5f6e066292cd440e9e3abb8d40047ada (table test-table [id=92b66a1c99914dbf8064a40cbc3c87fa]) successfully deleted
W20260811 19:04:11.889580 1791 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.109.1:46367 (error 108)
I20260811 19:04:11.890630 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 2020
I20260811 19:04:11.890872 2324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:11.890487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.1:46367 (local address 127.0.0.1:34736)
0811 19:04:11.890661 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:11.890756 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.1:46367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
W20260811 19:04:11.897815 1791 catalog_manager.cc:4584] TS ddd5a05b73cd4c46ab10a1588c743efb (127.13.109.1:46367): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5f6e066292cd440e9e3abb8d40047ada: Network error: Client connection negotiation failed: client connection to 127.13.109.1:46367: connect: Connection refused (error 111)
I20260811 19:04:11.906046 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 2166
I20260811 19:04:11.915417 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 1775
2026-08-11T19:04:11Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1177 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:04:11 dist-test-slave-0kls krb5kdc[2359](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:04:11 dist-test-slave-0kls krb5kdc[2359](info): set up 2 sockets
Aug 11 19:04:11 dist-test-slave-0kls krb5kdc[2359](info): commencing operation
krb5kdc: starting...
W20260811 19:04:13.027554 13748 mini_kdc.cc:121] Time spent starting KDC: real 1.087s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/krb5kdc/test-user.keytab.
Aug 11 19:04:13 dist-test-slave-0kls krb5kdc[2359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475053, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:04:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:04:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.109.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.109.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.109.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab.
Entry for principal kudu/127.13.109.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab.
I20260811 19:04:13.169275 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.62:36319
--webserver_interface=127.13.109.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab
--principal=kudu/127.13.109.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:46125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.109.62:36319
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:04:13.288703 2372 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:04:13.289095 2372 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:04:13.289189 2372 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:04:13.293566 2372 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:04:13.293726 2372 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:04:13.293766 2372 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:04:13.293792 2372 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:04:13.293843 2372 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:04:13.299953 2372 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:46125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.109.62:36319
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab
--principal=kudu/127.13.109.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.62:36319
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.109.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2372
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:04:13.301366 2372 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:04:13.302579 2372 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:04:13.311981 2380 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:04:13.311981 2377 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:04:13.312040 2378 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:04:13.312183 2372 server_base.cc:1034] running on GCE node
I20260811 19:04:13.313267 2372 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:04:13.314585 2372 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:04:13.315799 2372 hybrid_clock.cc:648] HybridClock initialized: now 1786475053315779 us; error 34 us; skew 500 ppm
Aug 11 19:04:13 dist-test-slave-0kls krb5kdc[2359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475053, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:04:13.318780 2372 init.cc:377] Logged in from keytab as kudu/127.13.109.62@KRBTEST.COM (short username kudu)
I20260811 19:04:13.319803 2372 webserver.cc:460] Webserver started at http://127.13.109.62:35387/ using document root <none> and password file <none>
I20260811 19:04:13.320425 2372 fs_manager.cc:359] Metadata directory not provided
I20260811 19:04:13.320529 2372 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:04:13.320796 2372 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:04:13.323110 2372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/minicluster-data/master-0/data/instance:
uuid: "193ff6c6619a4ee78295631a2293cb13"
format_stamp: "Formatted at 2026-08-11 19:04:13 on dist-test-slave-0kls"
server_key: "beaf5836662a90294eef9dac5bfd83db"
server_key_iv: "8d0a097588bbdc04a30330e843c737f9"
server_key_version: "encryptionkey@0"
I20260811 19:04:13.323765 2372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/minicluster-data/master-0/wal/instance:
uuid: "193ff6c6619a4ee78295631a2293cb13"
format_stamp: "Formatted at 2026-08-11 19:04:13 on dist-test-slave-0kls"
server_key: "beaf5836662a90294eef9dac5bfd83db"
server_key_iv: "8d0a097588bbdc04a30330e843c737f9"
server_key_version: "encryptionkey@0"
I20260811 19:04:13.328042 2372 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 19:04:13.330705 2389 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:04:13.332029 2372 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260811 19:04:13.332212 2372 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/minicluster-data/master-0/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/minicluster-data/master-0/wal
uuid: "193ff6c6619a4ee78295631a2293cb13"
format_stamp: "Formatted at 2026-08-11 19:04:13 on dist-test-slave-0kls"
server_key: "beaf5836662a90294eef9dac5bfd83db"
server_key_iv: "8d0a097588bbdc04a30330e843c737f9"
server_key_version: "encryptionkey@0"
I20260811 19:04:13.332351 2372 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:04:13.348181 2372 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:04:13.351259 2372 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:04:13.351500 2372 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:04:13.359414 2372 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.62:36319
I20260811 19:04:13.359443 2447 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.62:36319 every 8 connection(s)
I20260811 19:04:13.360483 2372 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
I20260811 19:04:13.363526 2448 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:04:13.367703 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 2372
I20260811 19:04:13.367851 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/minicluster-data/master-0/wal/instance
I20260811 19:04:13.368160 13748 external_mini_cluster.cc:1428] Setting key 9485721c4c00ba0364c5b78671d7a9f1
I20260811 19:04:13.369953 2448 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 193ff6c6619a4ee78295631a2293cb13: Bootstrap starting.
I20260811 19:04:13.372356 2448 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 193ff6c6619a4ee78295631a2293cb13: Neither blocks nor log segments found. Creating new log.
I20260811 19:04:13.373170 2448 log.cc:824] T 00000000000000000000000000000000 P 193ff6c6619a4ee78295631a2293cb13: Log is configured to *not* fsync() on all Append() calls
I20260811 19:04:13.375305 2448 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 193ff6c6619a4ee78295631a2293cb13: No bootstrap required, opened a new log
Aug 11 19:04:13 dist-test-slave-0kls krb5kdc[2359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475053, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
I20260811 19:04:13.379828 2448 raft_consensus.cc:348] T 00000000000000000000000000000000 P 193ff6c6619a4ee78295631a2293cb13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "193ff6c6619a4ee78295631a2293cb13" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 36319 } }
I20260811 19:04:13.380117 2448 raft_consensus.cc:374] T 00000000000000000000000000000000 P 193ff6c6619a4ee78295631a2293cb13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:04:13.380204 2448 raft_consensus.cc:729] T 00000000000000000000000000000000 P 193ff6c6619a4ee78295631a2293cb13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 193ff6c6619a4ee78295631a2293cb13, State: Initialized, Role: FOLLOWER
I20260811 19:04:13.380720 2448 consensus_queue.cc:260] T 00000000000000000000000000000000 P 193ff6c6619a4ee78295631a2293cb13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "193ff6c6619a4ee78295631a2293cb13" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 36319 } }
I20260811 19:04:13.380877 2448 raft_consensus.cc:388] T 00000000000000000000000000000000 P 193ff6c6619a4ee78295631a2293cb13 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:04:13.380949 2448 raft_consensus.cc:482] T 00000000000000000000000000000000 P 193ff6c6619a4ee78295631a2293cb13 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:04:13.381078 2448 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 193ff6c6619a4ee78295631a2293cb13 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:04:13.382134 2448 raft_consensus.cc:504] T 00000000000000000000000000000000 P 193ff6c6619a4ee78295631a2293cb13 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "193ff6c6619a4ee78295631a2293cb13" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 36319 } }
I20260811 19:04:13.382504 2448 leader_election.cc:302] T 00000000000000000000000000000000 P 193ff6c6619a4ee78295631a2293cb13 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 193ff6c6619a4ee78295631a2293cb13; no voters:
I20260811 19:04:13.382905 2448 leader_election.cc:288] T 00000000000000000000000000000000 P 193ff6c6619a4ee78295631a2293cb13 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:04:13.383015 2454 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 193ff6c6619a4ee78295631a2293cb13 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:04:13.383239 2454 raft_consensus.cc:686] T 00000000000000000000000000000000 P 193ff6c6619a4ee78295631a2293cb13 [term 1 LEADER]: Becoming Leader. State: Replica: 193ff6c6619a4ee78295631a2293cb13, State: Running, Role: LEADER
I20260811 19:04:13.383649 2454 consensus_queue.cc:237] T 00000000000000000000000000000000 P 193ff6c6619a4ee78295631a2293cb13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "193ff6c6619a4ee78295631a2293cb13" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 36319 } }
I20260811 19:04:13.384057 2452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:13.369814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34694 (local address 127.13.109.62:36319)
0811 19:04:13.370323 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:13.370339 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:13.370379 (+ 40us) server_negotiation.cc:407] Connection header received
0811 19:04:13.371090 (+ 711us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:13.371109 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:13.371471 (+ 362us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:13.371847 (+ 376us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:13.372754 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:13.373923 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:13.374824 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:13.375380 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:13.375560 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:13.378421 (+ 2861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:04:13.378427 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:04:13.378441 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:04:13.378469 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:04:13.380477 (+ 2008us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:13.381009 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:13.381014 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:13.381019 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:13.381203 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:13.381495 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:13.381499 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:13.381501 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:13.382058 (+ 557us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:04:13.382287 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:13.382601 (+ 314us) server_negotiation.cc:299] Negotiation successful
0811 19:04:13.382971 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":147,"threads_started":1}
I20260811 19:04:13.384349 2448 sys_catalog.cc:564] T 00000000000000000000000000000000 P 193ff6c6619a4ee78295631a2293cb13 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:04:13.387528 2456 sys_catalog.cc:455] T 00000000000000000000000000000000 P 193ff6c6619a4ee78295631a2293cb13 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 193ff6c6619a4ee78295631a2293cb13. Latest consensus state: current_term: 1 leader_uuid: "193ff6c6619a4ee78295631a2293cb13" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "193ff6c6619a4ee78295631a2293cb13" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 36319 } } }
I20260811 19:04:13.387529 2455 sys_catalog.cc:455] T 00000000000000000000000000000000 P 193ff6c6619a4ee78295631a2293cb13 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "193ff6c6619a4ee78295631a2293cb13" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "193ff6c6619a4ee78295631a2293cb13" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 36319 } } }
I20260811 19:04:13.387705 2456 sys_catalog.cc:458] T 00000000000000000000000000000000 P 193ff6c6619a4ee78295631a2293cb13 [sys.catalog]: This master's current role is: LEADER
I20260811 19:04:13.387768 2455 sys_catalog.cc:458] T 00000000000000000000000000000000 P 193ff6c6619a4ee78295631a2293cb13 [sys.catalog]: This master's current role is: LEADER
W20260811 19:04:13.389920 2469 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 193ff6c6619a4ee78295631a2293cb13: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 19:04:13.390012 2469 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 19:04:13.390087 2471 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:04:13.391070 2471 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:04:13.397294 2471 catalog_manager.cc:1347] Generated new cluster ID: 132ebcdc31af45509faa9d726c61ccd3
I20260811 19:04:13.397394 2471 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:04:13.421957 2471 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:04:13.423064 2471 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:04:13.427992 2471 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 193ff6c6619a4ee78295631a2293cb13: Generated new TSK 0
I20260811 19:04:13.428833 2471 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.109.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.109.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.109.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.1.keytab.
Entry for principal kudu/127.13.109.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.1.keytab.
I20260811 19:04:13.465131 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.1:0
--local_ip_for_outbound_sockets=127.13.109.1
--webserver_interface=127.13.109.1
--webserver_port=0
--tserver_master_addrs=127.13.109.62:36319
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.1.keytab
--principal=kudu/127.13.109.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:46125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:04:13.589202 2476 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:04:13.589500 2476 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:04:13.589546 2476 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:04:13.593822 2476 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:04:13.593894 2476 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:04:13.593955 2476 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.1
I20260811 19:04:13.599473 2476 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:46125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.1.keytab
--principal=kudu/127.13.109.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.109.1
--webserver_port=0
--tserver_master_addrs=127.13.109.62:36319
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2476
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.1
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:04:13.600678 2476 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:04:13.601892 2476 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:04:13.612413 2482 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:04:13.612466 2481 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:04:13.612689 2484 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:04:13.612787 2476 server_base.cc:1034] running on GCE node
I20260811 19:04:13.613340 2476 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:04:13.614279 2476 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:04:13.615463 2476 hybrid_clock.cc:648] HybridClock initialized: now 1786475053615443 us; error 39 us; skew 500 ppm
Aug 11 19:04:13 dist-test-slave-0kls krb5kdc[2359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475053, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:04:13.618427 2476 init.cc:377] Logged in from keytab as kudu/127.13.109.1@KRBTEST.COM (short username kudu)
I20260811 19:04:13.619400 2476 webserver.cc:460] Webserver started at http://127.13.109.1:42667/ using document root <none> and password file <none>
I20260811 19:04:13.620033 2476 fs_manager.cc:359] Metadata directory not provided
I20260811 19:04:13.620098 2476 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:04:13.620347 2476 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:04:13.622574 2476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/minicluster-data/ts-0/data/instance:
uuid: "97715af0378444829a4d367bcd618e94"
format_stamp: "Formatted at 2026-08-11 19:04:13 on dist-test-slave-0kls"
server_key: "152194cd327c29c514ccc025c9fc0ed8"
server_key_iv: "36f0dd3deb46b36ff66085c1fb937998"
server_key_version: "encryptionkey@0"
I20260811 19:04:13.623174 2476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/minicluster-data/ts-0/wal/instance:
uuid: "97715af0378444829a4d367bcd618e94"
format_stamp: "Formatted at 2026-08-11 19:04:13 on dist-test-slave-0kls"
server_key: "152194cd327c29c514ccc025c9fc0ed8"
server_key_iv: "36f0dd3deb46b36ff66085c1fb937998"
server_key_version: "encryptionkey@0"
I20260811 19:04:13.627053 2476 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 19:04:13.629703 2491 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:04:13.630992 2476 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 19:04:13.631136 2476 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/minicluster-data/ts-0/wal
uuid: "97715af0378444829a4d367bcd618e94"
format_stamp: "Formatted at 2026-08-11 19:04:13 on dist-test-slave-0kls"
server_key: "152194cd327c29c514ccc025c9fc0ed8"
server_key_iv: "36f0dd3deb46b36ff66085c1fb937998"
server_key_version: "encryptionkey@0"
I20260811 19:04:13.631273 2476 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:04:13.654064 2476 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:04:13.657455 2476 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:04:13.657699 2476 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:04:13.658409 2476 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:04:13.659369 2476 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:04:13.659449 2476 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:04:13.659526 2476 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:04:13.659574 2476 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:04:13.669209 2476 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.1:39915
I20260811 19:04:13.669229 2616 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.1:39915 every 8 connection(s)
I20260811 19:04:13.670258 2476 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
I20260811 19:04:13.673677 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 2476
I20260811 19:04:13.673864 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/minicluster-data/ts-0/wal/instance
I20260811 19:04:13.674176 13748 external_mini_cluster.cc:1428] Setting key 3f0bbee7185603ef3ee6ea0fe3d624f2
Aug 11 19:04:13 dist-test-slave-0kls krb5kdc[2359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475053, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.1@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.109.2@KRBTEST.COM; defaulting to no policy
I20260811 19:04:13.686342 2452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:13.672270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.1:55729 (local address 127.13.109.62:36319)
0811 19:04:13.672422 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:13.672426 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:13.673378 (+ 952us) server_negotiation.cc:407] Connection header received
0811 19:04:13.675098 (+ 1720us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:13.675101 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:13.675152 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:13.675229 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:13.676693 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:13.677539 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:13.678816 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:13.679056 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:13.679189 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:13.682221 (+ 3032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:04:13.682225 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:04:13.682239 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:04:13.682269 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:04:13.683818 (+ 1549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:13.684410 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:13.684418 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:13.684420 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:13.684499 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:13.684951 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:13.684956 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:13.684959 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:13.685179 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:04:13.685329 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:13.685987 (+ 658us) server_negotiation.cc:299] Negotiation successful
0811 19:04:13.686138 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 19:04:13.687291 2620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:13.672673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:36319 (local address 127.13.109.1:55729)
0811 19:04:13.673189 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:13.673236 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 19:04:13.674850 (+ 1614us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:04:13.675452 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:04:13.675461 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:04:13.676001 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:04:13.676515 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:13.676529 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:13.677674 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:13.677677 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:13.678677 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:13.678688 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:13.679237 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:13.679241 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:13.679423 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:13.680089 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:04:13.680111 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:04:13.682003 (+ 1892us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:04:13.683959 (+ 1956us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:04:13.683965 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:04:13.683985 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:04:13.684283 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:04:13.684607 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:04:13.684611 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:04:13.684613 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:04:13.684834 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:04:13.685306 (+ 472us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:04:13.685314 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:04:13.685678 (+ 364us) client_negotiation.cc:769] Sending connection context
0811 19:04:13.685926 (+ 248us) client_negotiation.cc:240] Negotiation successful
0811 19:04:13.686301 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":137,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.109.2@KRBTEST.COM" created.
I20260811 19:04:13.688788 2617 heartbeater.cc:347] Connected to a master server at 127.13.109.62:36319
I20260811 19:04:13.689024 2617 heartbeater.cc:464] Registering TS with master...
I20260811 19:04:13.689532 2617 heartbeater.cc:511] Master 127.13.109.62:36319 requested a full tablet report, sending...
I20260811 19:04:13.691208 2408 ts_manager.cc:194] Registered new tserver with Master: 97715af0378444829a4d367bcd618e94 (127.13.109.1:39915)
I20260811 19:04:13.692418 2408 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.109.1@KRBTEST.COM'} at 127.13.109.1:55729
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.109.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.2.keytab.
Entry for principal kudu/127.13.109.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.2.keytab.
I20260811 19:04:13.705878 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.2:0
--local_ip_for_outbound_sockets=127.13.109.2
--webserver_interface=127.13.109.2
--webserver_port=0
--tserver_master_addrs=127.13.109.62:36319
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.2.keytab
--principal=kudu/127.13.109.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:46125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:04:13.824651 2623 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:04:13.824941 2623 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:04:13.824988 2623 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:04:13.829231 2623 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:04:13.829314 2623 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:04:13.829376 2623 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.2
I20260811 19:04:13.835114 2623 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:46125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.2.keytab
--principal=kudu/127.13.109.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.109.2
--webserver_port=0
--tserver_master_addrs=127.13.109.62:36319
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2623
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.2
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:04:13.836431 2623 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:04:13.837515 2623 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:04:13.848049 2631 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:04:13.848090 2628 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:04:13.848313 2623 server_base.cc:1034] running on GCE node
W20260811 19:04:13.848213 2629 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:04:13.848949 2623 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:04:13.849668 2623 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:04:13.850840 2623 hybrid_clock.cc:648] HybridClock initialized: now 1786475053850824 us; error 26 us; skew 500 ppm
Aug 11 19:04:13 dist-test-slave-0kls krb5kdc[2359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475053, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:04:13.853827 2623 init.cc:377] Logged in from keytab as kudu/127.13.109.2@KRBTEST.COM (short username kudu)
I20260811 19:04:13.854883 2623 webserver.cc:460] Webserver started at http://127.13.109.2:41205/ using document root <none> and password file <none>
I20260811 19:04:13.855527 2623 fs_manager.cc:359] Metadata directory not provided
I20260811 19:04:13.855634 2623 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:04:13.855904 2623 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:04:13.858289 2623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/minicluster-data/ts-1/data/instance:
uuid: "988811a677384142bc818cc6fdbe300a"
format_stamp: "Formatted at 2026-08-11 19:04:13 on dist-test-slave-0kls"
server_key: "72e1c4226b67528ac3f62df0f3ffe5a2"
server_key_iv: "101985ebb195c297ba5dc934abf29af3"
server_key_version: "encryptionkey@0"
I20260811 19:04:13.858911 2623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/minicluster-data/ts-1/wal/instance:
uuid: "988811a677384142bc818cc6fdbe300a"
format_stamp: "Formatted at 2026-08-11 19:04:13 on dist-test-slave-0kls"
server_key: "72e1c4226b67528ac3f62df0f3ffe5a2"
server_key_iv: "101985ebb195c297ba5dc934abf29af3"
server_key_version: "encryptionkey@0"
I20260811 19:04:13.863124 2623 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260811 19:04:13.866047 2639 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:04:13.867447 2623 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 19:04:13.867616 2623 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/minicluster-data/ts-1/wal
uuid: "988811a677384142bc818cc6fdbe300a"
format_stamp: "Formatted at 2026-08-11 19:04:13 on dist-test-slave-0kls"
server_key: "72e1c4226b67528ac3f62df0f3ffe5a2"
server_key_iv: "101985ebb195c297ba5dc934abf29af3"
server_key_version: "encryptionkey@0"
I20260811 19:04:13.867767 2623 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:04:13.879248 2623 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:04:13.882802 2623 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:04:13.883080 2623 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:04:13.883838 2623 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:04:13.884858 2623 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:04:13.884919 2623 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:04:13.884964 2623 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:04:13.884979 2623 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:04:13.895432 2623 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.2:39669
I20260811 19:04:13.895454 2764 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.2:39669 every 8 connection(s)
I20260811 19:04:13.896493 2623 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
I20260811 19:04:13.904295 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 2623
I20260811 19:04:13.904491 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/minicluster-data/ts-1/wal/instance
I20260811 19:04:13.904846 13748 external_mini_cluster.cc:1428] Setting key 58cbee08414d78a0e9dc07dad9d5cf88
Aug 11 19:04:13 dist-test-slave-0kls krb5kdc[2359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475053, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.2@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
I20260811 19:04:13.912751 2452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:13.898547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.2:53351 (local address 127.13.109.62:36319)
0811 19:04:13.898707 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:13.898711 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:13.900124 (+ 1413us) server_negotiation.cc:407] Connection header received
0811 19:04:13.901051 (+ 927us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:13.901054 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:13.901107 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:13.901196 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:13.902630 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:13.903173 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:13.904394 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:13.904751 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:13.904869 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:13.908334 (+ 3465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:04:13.908338 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:04:13.908340 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:04:13.908367 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:04:13.910188 (+ 1821us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:13.910845 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:13.910852 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:13.910856 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:13.910963 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:13.911455 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:13.911457 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:13.911459 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:13.911668 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:04:13.911754 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:13.912489 (+ 735us) server_negotiation.cc:299] Negotiation successful
0811 19:04:13.912616 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 19:04:13.914229 2769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:13.898968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:36319 (local address 127.13.109.2:53351)
0811 19:04:13.899930 (+ 962us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:13.899984 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 19:04:13.900825 (+ 841us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:04:13.901394 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:04:13.901402 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:04:13.901880 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:04:13.902455 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:13.902479 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:13.903306 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:13.903309 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:13.904222 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:13.904242 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:13.904938 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:13.904943 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:13.905131 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:13.905948 (+ 817us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:04:13.905971 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:04:13.908112 (+ 2141us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:04:13.910364 (+ 2252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:04:13.910371 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:04:13.910390 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:04:13.910715 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:04:13.911094 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:04:13.911101 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:04:13.911106 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:04:13.911343 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:04:13.911795 (+ 452us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:04:13.911806 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:04:13.912205 (+ 399us) client_negotiation.cc:769] Sending connection context
0811 19:04:13.912448 (+ 243us) client_negotiation.cc:240] Negotiation successful
0811 19:04:13.912916 (+ 468us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":733,"thread_start_us":582,"threads_started":1}
I20260811 19:04:13.915657 2766 heartbeater.cc:347] Connected to a master server at 127.13.109.62:36319
I20260811 19:04:13.915941 2766 heartbeater.cc:464] Registering TS with master...
I20260811 19:04:13.916608 2766 heartbeater.cc:511] Master 127.13.109.62:36319 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.13.109.3@KRBTEST.COM; defaulting to no policy
I20260811 19:04:13.918071 2408 ts_manager.cc:194] Registered new tserver with Master: 988811a677384142bc818cc6fdbe300a (127.13.109.2:39669)
I20260811 19:04:13.918684 2408 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.109.2@KRBTEST.COM'} at 127.13.109.2:53351
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.109.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.109.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.3.keytab.
Entry for principal kudu/127.13.109.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.3.keytab.
I20260811 19:04:13.939111 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.3:0
--local_ip_for_outbound_sockets=127.13.109.3
--webserver_interface=127.13.109.3
--webserver_port=0
--tserver_master_addrs=127.13.109.62:36319
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.3.keytab
--principal=kudu/127.13.109.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:46125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:04:14.069410 2773 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:04:14.069774 2773 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:04:14.069878 2773 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:04:14.074338 2773 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:04:14.074440 2773 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:04:14.074533 2773 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.3
I20260811 19:04:14.080323 2773 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:46125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.3.keytab
--principal=kudu/127.13.109.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.109.3
--webserver_port=0
--tserver_master_addrs=127.13.109.62:36319
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2773
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.3
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:04:14.081533 2773 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:04:14.082685 2773 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:04:14.093508 2778 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:04:14.093493 2779 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:04:14.093493 2781 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:04:14.093616 2773 server_base.cc:1034] running on GCE node
I20260811 19:04:14.094313 2773 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:04:14.094965 2773 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:04:14.096135 2773 hybrid_clock.cc:648] HybridClock initialized: now 1786475054096096 us; error 51 us; skew 500 ppm
Aug 11 19:04:14 dist-test-slave-0kls krb5kdc[2359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475054, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:04:14.099104 2773 init.cc:377] Logged in from keytab as kudu/127.13.109.3@KRBTEST.COM (short username kudu)
I20260811 19:04:14.100112 2773 webserver.cc:460] Webserver started at http://127.13.109.3:41277/ using document root <none> and password file <none>
I20260811 19:04:14.100708 2773 fs_manager.cc:359] Metadata directory not provided
I20260811 19:04:14.100806 2773 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:04:14.101063 2773 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:04:14.103410 2773 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/minicluster-data/ts-2/data/instance:
uuid: "d8a00218b70349268f66766b52fca693"
format_stamp: "Formatted at 2026-08-11 19:04:14 on dist-test-slave-0kls"
server_key: "6a7431c01ab6f09de0f3ab892c6b45b8"
server_key_iv: "ed71254fbb4000dfca10c0a92eeeb10b"
server_key_version: "encryptionkey@0"
I20260811 19:04:14.104033 2773 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/minicluster-data/ts-2/wal/instance:
uuid: "d8a00218b70349268f66766b52fca693"
format_stamp: "Formatted at 2026-08-11 19:04:14 on dist-test-slave-0kls"
server_key: "6a7431c01ab6f09de0f3ab892c6b45b8"
server_key_iv: "ed71254fbb4000dfca10c0a92eeeb10b"
server_key_version: "encryptionkey@0"
I20260811 19:04:14.108038 2773 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 19:04:14.110602 2788 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:04:14.111932 2773 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 19:04:14.112110 2773 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/minicluster-data/ts-2/wal
uuid: "d8a00218b70349268f66766b52fca693"
format_stamp: "Formatted at 2026-08-11 19:04:14 on dist-test-slave-0kls"
server_key: "6a7431c01ab6f09de0f3ab892c6b45b8"
server_key_iv: "ed71254fbb4000dfca10c0a92eeeb10b"
server_key_version: "encryptionkey@0"
I20260811 19:04:14.112231 2773 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:04:14.129557 2773 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:04:14.133185 2773 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:04:14.133370 2773 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:04:14.134085 2773 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:04:14.135006 2773 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:04:14.135113 2773 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:04:14.135210 2773 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:04:14.135265 2773 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:04:14.145138 2773 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.3:42495
I20260811 19:04:14.145164 2916 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.3:42495 every 8 connection(s)
I20260811 19:04:14.146166 2773 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
I20260811 19:04:14.148000 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 2773
I20260811 19:04:14.148144 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/minicluster-data/ts-2/wal/instance
I20260811 19:04:14.148524 13748 external_mini_cluster.cc:1428] Setting key 405e1bea309cdab7cad981a306416f92
Aug 11 19:04:14 dist-test-slave-0kls krb5kdc[2359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475054, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.3@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
I20260811 19:04:14.160517 2452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:14.148170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.3:59793 (local address 127.13.109.62:36319)
0811 19:04:14.148380 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:14.148385 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:14.149257 (+ 872us) server_negotiation.cc:407] Connection header received
0811 19:04:14.150999 (+ 1742us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:14.151001 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:14.151051 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:14.151117 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:14.152299 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:14.152836 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:14.154107 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:14.154485 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:14.154575 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:14.156980 (+ 2405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:04:14.156983 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:04:14.156985 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:04:14.157010 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:04:14.158612 (+ 1602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:14.159080 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:14.159083 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:14.159084 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:14.159160 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:14.159474 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:14.159476 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:14.159478 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:14.159661 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:04:14.159754 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:14.160276 (+ 522us) server_negotiation.cc:299] Negotiation successful
0811 19:04:14.160387 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260811 19:04:14.161479 2919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:14.148528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:36319 (local address 127.13.109.3:59793)
0811 19:04:14.149074 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:14.149134 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 19:04:14.150773 (+ 1639us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:04:14.151281 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:04:14.151287 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:04:14.151648 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:04:14.152158 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:14.152172 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:14.152948 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:14.152951 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:14.153988 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:14.153999 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:14.154631 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:14.154634 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:14.154801 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:14.155350 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:04:14.155367 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:04:14.156826 (+ 1459us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:04:14.158710 (+ 1884us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:04:14.158715 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:04:14.158733 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:04:14.158980 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:04:14.159250 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:04:14.159253 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:04:14.159254 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:04:14.159387 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:04:14.159777 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:04:14.159782 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:04:14.160020 (+ 238us) client_negotiation.cc:769] Sending connection context
0811 19:04:14.160187 (+ 167us) client_negotiation.cc:240] Negotiation successful
0811 19:04:14.160449 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":122,"threads_started":1}
I20260811 19:04:14.162822 2917 heartbeater.cc:347] Connected to a master server at 127.13.109.62:36319
I20260811 19:04:14.163090 2917 heartbeater.cc:464] Registering TS with master...
I20260811 19:04:14.163637 2917 heartbeater.cc:511] Master 127.13.109.62:36319 requested a full tablet report, sending...
I20260811 19:04:14.164825 2408 ts_manager.cc:194] Registered new tserver with Master: d8a00218b70349268f66766b52fca693 (127.13.109.3:42495)
I20260811 19:04:14.165411 2408 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.109.3@KRBTEST.COM'} at 127.13.109.3:59793
I20260811 19:04:14.174810 13748 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 19:04:14.184094 2452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:14.176888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34696 (local address 127.13.109.62:36319)
0811 19:04:14.177039 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:14.177044 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:14.177236 (+ 192us) server_negotiation.cc:407] Connection header received
0811 19:04:14.177336 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:14.177338 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:14.177380 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:14.177439 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:14.178176 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:14.178677 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:14.179471 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:14.179670 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:14.179772 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:14.181118 (+ 1346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:04:14.181120 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:04:14.181122 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:04:14.181142 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:04:14.182439 (+ 1297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:14.182820 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:14.182822 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:14.182823 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:14.182897 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:14.183129 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:14.183131 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:14.183132 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:14.183347 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:04:14.183454 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:14.183759 (+ 305us) server_negotiation.cc:299] Negotiation successful
0811 19:04:14.183858 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 19:04:14.187276 2408 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34696:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 19:04:14.190455 2408 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 19:04:14.204919 2929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:14.199026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.1:39915 (local address 127.0.0.1:38656)
0811 19:04:14.199367 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:14.199442 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0811 19:04:14.199571 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:04:14.200329 (+ 758us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:04:14.200335 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:04:14.200356 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:04:14.200511 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:14.200519 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:14.201823 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:14.201826 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:14.203191 (+ 1365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:14.203200 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:14.204329 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:14.204333 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:14.204447 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:14.204505 (+ 58us) client_negotiation.cc:769] Sending connection context
0811 19:04:14.204615 (+ 110us) client_negotiation.cc:240] Negotiation successful
0811 19:04:14.204758 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":89,"threads_started":1}
I20260811 19:04:14.205186 2930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:14.199228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38656 (local address 127.13.109.1:39915)
0811 19:04:14.199577 (+ 349us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:14.199584 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:14.199610 (+ 26us) server_negotiation.cc:407] Connection header received
0811 19:04:14.199731 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:14.199755 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:14.199975 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:14.200132 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:04:14.200645 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:14.201684 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:14.203341 (+ 1657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:14.204174 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:14.204320 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:14.204496 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:14.204807 (+ 311us) server_negotiation.cc:299] Negotiation successful
0811 19:04:14.204910 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":245,"thread_start_us":138,"threads_started":1}
I20260811 19:04:14.205406 2932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:14.199792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.2:39669 (local address 127.0.0.1:44722)
0811 19:04:14.200645 (+ 853us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:14.200657 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 19:04:14.200733 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:04:14.201533 (+ 800us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:04:14.201536 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:04:14.201559 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:04:14.201652 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:14.201658 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:14.202667 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:14.202670 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:14.203686 (+ 1016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:14.203695 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:14.204579 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:14.204583 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:14.204672 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:14.204696 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 19:04:14.204785 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 19:04:14.204853 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":786,"thread_start_us":142,"threads_started":1}
I20260811 19:04:14.205626 2934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:14.200266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44722 (local address 127.13.109.2:39669)
0811 19:04:14.200987 (+ 721us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:14.201003 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:14.201022 (+ 19us) server_negotiation.cc:407] Connection header received
0811 19:04:14.201085 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:14.201089 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:14.201239 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:14.201367 (+ 128us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:04:14.201814 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:14.202526 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:14.203821 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:14.204448 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:14.204550 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:14.204713 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:14.205000 (+ 287us) server_negotiation.cc:299] Negotiation successful
0811 19:04:14.205380 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":609,"thread_start_us":120,"threads_started":1}
I20260811 19:04:14.206180 2931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:14.199913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.3:42495 (local address 127.0.0.1:55076)
0811 19:04:14.200791 (+ 878us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:14.200805 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 19:04:14.200904 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:04:14.201359 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:04:14.201362 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:04:14.201380 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:04:14.201477 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:14.201482 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:14.203085 (+ 1603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:14.203088 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:14.204224 (+ 1136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:14.204237 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:14.205533 (+ 1296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:14.205542 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:14.205644 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:14.205669 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 19:04:14.205768 (+ 99us) client_negotiation.cc:240] Negotiation successful
0811 19:04:14.205861 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":817,"thread_start_us":112,"threads_started":1}
I20260811 19:04:14.206597 2933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:14.200115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55076 (local address 127.13.109.3:42495)
0811 19:04:14.200529 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:14.200548 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:14.200923 (+ 375us) server_negotiation.cc:407] Connection header received
0811 19:04:14.201012 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:14.201018 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:14.201199 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:14.201362 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:04:14.201591 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:14.202764 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:14.204368 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:14.205226 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:14.205332 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:14.206313 (+ 981us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:14.206371 (+ 58us) server_negotiation.cc:299] Negotiation successful
0811 19:04:14.206462 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":104,"threads_started":1}
I20260811 19:04:14.208493 2543 tablet_service.cc:1467] Processing CreateTablet for tablet 9dc80b0c541d4a30a89eebae5ebcae61 (DEFAULT_TABLE table=test-table [id=39665d33f22e4bfc998ca024ff5454f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:04:14.208707 2691 tablet_service.cc:1467] Processing CreateTablet for tablet 9dc80b0c541d4a30a89eebae5ebcae61 (DEFAULT_TABLE table=test-table [id=39665d33f22e4bfc998ca024ff5454f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:04:14.209121 2843 tablet_service.cc:1467] Processing CreateTablet for tablet 9dc80b0c541d4a30a89eebae5ebcae61 (DEFAULT_TABLE table=test-table [id=39665d33f22e4bfc998ca024ff5454f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:04:14.209674 2543 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9dc80b0c541d4a30a89eebae5ebcae61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:04:14.209836 2691 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9dc80b0c541d4a30a89eebae5ebcae61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:04:14.210163 2843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9dc80b0c541d4a30a89eebae5ebcae61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:04:14.217134 2935 tablet_bootstrap.cc:492] T 9dc80b0c541d4a30a89eebae5ebcae61 P 97715af0378444829a4d367bcd618e94: Bootstrap starting.
I20260811 19:04:14.217885 2937 tablet_bootstrap.cc:492] T 9dc80b0c541d4a30a89eebae5ebcae61 P d8a00218b70349268f66766b52fca693: Bootstrap starting.
I20260811 19:04:14.219466 2938 tablet_bootstrap.cc:492] T 9dc80b0c541d4a30a89eebae5ebcae61 P 988811a677384142bc818cc6fdbe300a: Bootstrap starting.
I20260811 19:04:14.220093 2937 tablet_bootstrap.cc:654] T 9dc80b0c541d4a30a89eebae5ebcae61 P d8a00218b70349268f66766b52fca693: Neither blocks nor log segments found. Creating new log.
I20260811 19:04:14.220117 2935 tablet_bootstrap.cc:654] T 9dc80b0c541d4a30a89eebae5ebcae61 P 97715af0378444829a4d367bcd618e94: Neither blocks nor log segments found. Creating new log.
I20260811 19:04:14.221062 2935 log.cc:824] T 9dc80b0c541d4a30a89eebae5ebcae61 P 97715af0378444829a4d367bcd618e94: Log is configured to *not* fsync() on all Append() calls
I20260811 19:04:14.221407 2937 log.cc:824] T 9dc80b0c541d4a30a89eebae5ebcae61 P d8a00218b70349268f66766b52fca693: Log is configured to *not* fsync() on all Append() calls
I20260811 19:04:14.222266 2938 tablet_bootstrap.cc:654] T 9dc80b0c541d4a30a89eebae5ebcae61 P 988811a677384142bc818cc6fdbe300a: Neither blocks nor log segments found. Creating new log.
I20260811 19:04:14.223053 2938 log.cc:824] T 9dc80b0c541d4a30a89eebae5ebcae61 P 988811a677384142bc818cc6fdbe300a: Log is configured to *not* fsync() on all Append() calls
I20260811 19:04:14.223588 2935 tablet_bootstrap.cc:492] T 9dc80b0c541d4a30a89eebae5ebcae61 P 97715af0378444829a4d367bcd618e94: No bootstrap required, opened a new log
I20260811 19:04:14.223841 2935 ts_tablet_manager.cc:1397] T 9dc80b0c541d4a30a89eebae5ebcae61 P 97715af0378444829a4d367bcd618e94: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260811 19:04:14.224951 2938 tablet_bootstrap.cc:492] T 9dc80b0c541d4a30a89eebae5ebcae61 P 988811a677384142bc818cc6fdbe300a: No bootstrap required, opened a new log
I20260811 19:04:14.225139 2937 tablet_bootstrap.cc:492] T 9dc80b0c541d4a30a89eebae5ebcae61 P d8a00218b70349268f66766b52fca693: No bootstrap required, opened a new log
I20260811 19:04:14.225142 2938 ts_tablet_manager.cc:1397] T 9dc80b0c541d4a30a89eebae5ebcae61 P 988811a677384142bc818cc6fdbe300a: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 19:04:14.225415 2937 ts_tablet_manager.cc:1397] T 9dc80b0c541d4a30a89eebae5ebcae61 P d8a00218b70349268f66766b52fca693: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.006s
I20260811 19:04:14.228130 2938 raft_consensus.cc:348] T 9dc80b0c541d4a30a89eebae5ebcae61 P 988811a677384142bc818cc6fdbe300a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97715af0378444829a4d367bcd618e94" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 39915 } } peers { permanent_uuid: "988811a677384142bc818cc6fdbe300a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 39669 } } peers { permanent_uuid: "d8a00218b70349268f66766b52fca693" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 42495 } }
I20260811 19:04:14.228399 2938 raft_consensus.cc:374] T 9dc80b0c541d4a30a89eebae5ebcae61 P 988811a677384142bc818cc6fdbe300a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:04:14.228453 2938 raft_consensus.cc:729] T 9dc80b0c541d4a30a89eebae5ebcae61 P 988811a677384142bc818cc6fdbe300a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 988811a677384142bc818cc6fdbe300a, State: Initialized, Role: FOLLOWER
I20260811 19:04:14.228782 2935 raft_consensus.cc:348] T 9dc80b0c541d4a30a89eebae5ebcae61 P 97715af0378444829a4d367bcd618e94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97715af0378444829a4d367bcd618e94" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 39915 } } peers { permanent_uuid: "988811a677384142bc818cc6fdbe300a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 39669 } } peers { permanent_uuid: "d8a00218b70349268f66766b52fca693" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 42495 } }
I20260811 19:04:14.229048 2935 raft_consensus.cc:374] T 9dc80b0c541d4a30a89eebae5ebcae61 P 97715af0378444829a4d367bcd618e94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:04:14.228971 2938 consensus_queue.cc:260] T 9dc80b0c541d4a30a89eebae5ebcae61 P 988811a677384142bc818cc6fdbe300a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97715af0378444829a4d367bcd618e94" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 39915 } } peers { permanent_uuid: "988811a677384142bc818cc6fdbe300a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 39669 } } peers { permanent_uuid: "d8a00218b70349268f66766b52fca693" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 42495 } }
I20260811 19:04:14.229108 2935 raft_consensus.cc:729] T 9dc80b0c541d4a30a89eebae5ebcae61 P 97715af0378444829a4d367bcd618e94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97715af0378444829a4d367bcd618e94, State: Initialized, Role: FOLLOWER
I20260811 19:04:14.229382 2937 raft_consensus.cc:348] T 9dc80b0c541d4a30a89eebae5ebcae61 P d8a00218b70349268f66766b52fca693 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97715af0378444829a4d367bcd618e94" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 39915 } } peers { permanent_uuid: "988811a677384142bc818cc6fdbe300a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 39669 } } peers { permanent_uuid: "d8a00218b70349268f66766b52fca693" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 42495 } }
I20260811 19:04:14.229532 2935 consensus_queue.cc:260] T 9dc80b0c541d4a30a89eebae5ebcae61 P 97715af0378444829a4d367bcd618e94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97715af0378444829a4d367bcd618e94" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 39915 } } peers { permanent_uuid: "988811a677384142bc818cc6fdbe300a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 39669 } } peers { permanent_uuid: "d8a00218b70349268f66766b52fca693" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 42495 } }
I20260811 19:04:14.229650 2937 raft_consensus.cc:374] T 9dc80b0c541d4a30a89eebae5ebcae61 P d8a00218b70349268f66766b52fca693 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:04:14.229714 2937 raft_consensus.cc:729] T 9dc80b0c541d4a30a89eebae5ebcae61 P d8a00218b70349268f66766b52fca693 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8a00218b70349268f66766b52fca693, State: Initialized, Role: FOLLOWER
I20260811 19:04:14.230145 2938 ts_tablet_manager.cc:1428] T 9dc80b0c541d4a30a89eebae5ebcae61 P 988811a677384142bc818cc6fdbe300a: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260811 19:04:14.230268 2617 heartbeater.cc:503] Master 127.13.109.62:36319 was elected leader, sending a full tablet report...
I20260811 19:04:14.230190 2937 consensus_queue.cc:260] T 9dc80b0c541d4a30a89eebae5ebcae61 P d8a00218b70349268f66766b52fca693 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97715af0378444829a4d367bcd618e94" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 39915 } } peers { permanent_uuid: "988811a677384142bc818cc6fdbe300a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 39669 } } peers { permanent_uuid: "d8a00218b70349268f66766b52fca693" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 42495 } }
I20260811 19:04:14.230522 2766 heartbeater.cc:503] Master 127.13.109.62:36319 was elected leader, sending a full tablet report...
I20260811 19:04:14.231753 2935 ts_tablet_manager.cc:1428] T 9dc80b0c541d4a30a89eebae5ebcae61 P 97715af0378444829a4d367bcd618e94: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260811 19:04:14.231839 2917 heartbeater.cc:503] Master 127.13.109.62:36319 was elected leader, sending a full tablet report...
I20260811 19:04:14.232957 2937 ts_tablet_manager.cc:1428] T 9dc80b0c541d4a30a89eebae5ebcae61 P d8a00218b70349268f66766b52fca693: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
W20260811 19:04:14.397298 2918 tablet.cc:2369] T 9dc80b0c541d4a30a89eebae5ebcae61 P d8a00218b70349268f66766b52fca693: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 19:04:14.397805 2768 tablet.cc:2369] T 9dc80b0c541d4a30a89eebae5ebcae61 P 988811a677384142bc818cc6fdbe300a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 19:04:14.421974 2619 tablet.cc:2369] T 9dc80b0c541d4a30a89eebae5ebcae61 P 97715af0378444829a4d367bcd618e94: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 19:04:14.613438 2944 raft_consensus.cc:482] T 9dc80b0c541d4a30a89eebae5ebcae61 P 988811a677384142bc818cc6fdbe300a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:04:14.613730 2944 raft_consensus.cc:504] T 9dc80b0c541d4a30a89eebae5ebcae61 P 988811a677384142bc818cc6fdbe300a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97715af0378444829a4d367bcd618e94" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 39915 } } peers { permanent_uuid: "988811a677384142bc818cc6fdbe300a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 39669 } } peers { permanent_uuid: "d8a00218b70349268f66766b52fca693" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 42495 } }
I20260811 19:04:14.615099 2944 leader_election.cc:288] T 9dc80b0c541d4a30a89eebae5ebcae61 P 988811a677384142bc818cc6fdbe300a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97715af0378444829a4d367bcd618e94 (127.13.109.1:39915), d8a00218b70349268f66766b52fca693 (127.13.109.3:42495)
I20260811 19:04:14.620503 2946 raft_consensus.cc:482] T 9dc80b0c541d4a30a89eebae5ebcae61 P d8a00218b70349268f66766b52fca693 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:04:14.620820 2933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:14.615900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.2:47887 (local address 127.13.109.3:42495)
0811 19:04:14.616027 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:14.616032 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:14.616214 (+ 182us) server_negotiation.cc:407] Connection header received
0811 19:04:14.616470 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:14.616474 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:14.616537 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:14.616614 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:04:14.617193 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:14.617981 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:14.619485 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:14.620123 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:14.620258 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:14.620324 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:14.620611 (+ 287us) server_negotiation.cc:299] Negotiation successful
0811 19:04:14.620697 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 19:04:14.620877 2947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:14.615424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.3:42495 (local address 127.13.109.2:47887)
0811 19:04:14.616117 (+ 693us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:14.616131 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 19:04:14.616322 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:04:14.616812 (+ 490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:04:14.616856 (+ 44us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:04:14.616870 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:04:14.617037 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:14.617045 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:14.618210 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:14.618213 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:14.619348 (+ 1135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:14.619358 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:14.620273 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:14.620278 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:14.620387 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:14.620484 (+ 97us) client_negotiation.cc:769] Sending connection context
0811 19:04:14.620565 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 19:04:14.620711 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":587,"mutex_wait_us":98,"spinlock_wait_cycles":15616,"thread_start_us":97,"threads_started":1}
I20260811 19:04:14.620790 2946 raft_consensus.cc:504] T 9dc80b0c541d4a30a89eebae5ebcae61 P d8a00218b70349268f66766b52fca693 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97715af0378444829a4d367bcd618e94" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 39915 } } peers { permanent_uuid: "988811a677384142bc818cc6fdbe300a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 39669 } } peers { permanent_uuid: "d8a00218b70349268f66766b52fca693" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 42495 } }
I20260811 19:04:14.621232 2930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:14.616153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.2:46315 (local address 127.13.109.1:39915)
0811 19:04:14.616422 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:14.616430 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:14.616448 (+ 18us) server_negotiation.cc:407] Connection header received
0811 19:04:14.616589 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:14.616594 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:14.616701 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:14.616792 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:04:14.617264 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:14.618286 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:14.619589 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:14.620225 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:14.620322 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:14.620373 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:14.621035 (+ 662us) server_negotiation.cc:299] Negotiation successful
0811 19:04:14.621120 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146}
I20260811 19:04:14.621259 2949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:14.615360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.1:39915 (local address 127.13.109.2:46315)
0811 19:04:14.615757 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:14.615774 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 19:04:14.615911 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:04:14.616823 (+ 912us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:04:14.616826 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:04:14.616846 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:04:14.617077 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:14.617086 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:14.618442 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:14.618446 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:14.619457 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:14.619468 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:14.620327 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:14.620330 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:14.620475 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:14.620494 (+ 19us) client_negotiation.cc:769] Sending connection context
0811 19:04:14.620565 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 19:04:14.620802 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"mutex_wait_us":218,"thread_start_us":111,"threads_started":1}
I20260811 19:04:14.621932 2865 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dc80b0c541d4a30a89eebae5ebcae61" candidate_uuid: "988811a677384142bc818cc6fdbe300a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8a00218b70349268f66766b52fca693" is_pre_election: true
I20260811 19:04:14.621935 2567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dc80b0c541d4a30a89eebae5ebcae61" candidate_uuid: "988811a677384142bc818cc6fdbe300a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97715af0378444829a4d367bcd618e94" is_pre_election: true
I20260811 19:04:14.622174 2865 raft_consensus.cc:2454] T 9dc80b0c541d4a30a89eebae5ebcae61 P d8a00218b70349268f66766b52fca693 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 988811a677384142bc818cc6fdbe300a in term 0.
I20260811 19:04:14.622224 2567 raft_consensus.cc:2454] T 9dc80b0c541d4a30a89eebae5ebcae61 P 97715af0378444829a4d367bcd618e94 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 988811a677384142bc818cc6fdbe300a in term 0.
I20260811 19:04:14.622445 2946 leader_election.cc:288] T 9dc80b0c541d4a30a89eebae5ebcae61 P d8a00218b70349268f66766b52fca693 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97715af0378444829a4d367bcd618e94 (127.13.109.1:39915), 988811a677384142bc818cc6fdbe300a (127.13.109.2:39669)
I20260811 19:04:14.622747 2642 leader_election.cc:302] T 9dc80b0c541d4a30a89eebae5ebcae61 P 988811a677384142bc818cc6fdbe300a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 97715af0378444829a4d367bcd618e94, 988811a677384142bc818cc6fdbe300a; no voters:
I20260811 19:04:14.623059 2944 raft_consensus.cc:2781] T 9dc80b0c541d4a30a89eebae5ebcae61 P 988811a677384142bc818cc6fdbe300a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 19:04:14.623152 2944 raft_consensus.cc:482] T 9dc80b0c541d4a30a89eebae5ebcae61 P 988811a677384142bc818cc6fdbe300a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 19:04:14.623203 2944 raft_consensus.cc:3037] T 9dc80b0c541d4a30a89eebae5ebcae61 P 988811a677384142bc818cc6fdbe300a [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:04:14.624699 2944 raft_consensus.cc:504] T 9dc80b0c541d4a30a89eebae5ebcae61 P 988811a677384142bc818cc6fdbe300a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97715af0378444829a4d367bcd618e94" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 39915 } } peers { permanent_uuid: "988811a677384142bc818cc6fdbe300a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 39669 } } peers { permanent_uuid: "d8a00218b70349268f66766b52fca693" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 42495 } }
I20260811 19:04:14.625082 2944 leader_election.cc:288] T 9dc80b0c541d4a30a89eebae5ebcae61 P 988811a677384142bc818cc6fdbe300a [CANDIDATE]: Term 1 election: Requested vote from peers 97715af0378444829a4d367bcd618e94 (127.13.109.1:39915), d8a00218b70349268f66766b52fca693 (127.13.109.3:42495)
I20260811 19:04:14.625491 2865 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dc80b0c541d4a30a89eebae5ebcae61" candidate_uuid: "988811a677384142bc818cc6fdbe300a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8a00218b70349268f66766b52fca693"
I20260811 19:04:14.625630 2865 raft_consensus.cc:3037] T 9dc80b0c541d4a30a89eebae5ebcae61 P d8a00218b70349268f66766b52fca693 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:04:14.625931 2567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dc80b0c541d4a30a89eebae5ebcae61" candidate_uuid: "988811a677384142bc818cc6fdbe300a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97715af0378444829a4d367bcd618e94"
I20260811 19:04:14.626080 2567 raft_consensus.cc:3037] T 9dc80b0c541d4a30a89eebae5ebcae61 P 97715af0378444829a4d367bcd618e94 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:04:14.626968 2865 raft_consensus.cc:2454] T 9dc80b0c541d4a30a89eebae5ebcae61 P d8a00218b70349268f66766b52fca693 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 988811a677384142bc818cc6fdbe300a in term 1.
I20260811 19:04:14.627298 2919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:14.622638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.1:39915 (local address 127.13.109.3:37311)
0811 19:04:14.622778 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:14.622792 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 19:04:14.622870 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:04:14.623786 (+ 916us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:04:14.623788 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:04:14.623806 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:04:14.623896 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:14.623900 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:14.624719 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:14.624722 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:14.625922 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:14.625932 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:14.626874 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:14.626877 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:14.626983 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:14.627007 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 19:04:14.627081 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 19:04:14.627154 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260811 19:04:14.627317 2643 leader_election.cc:302] T 9dc80b0c541d4a30a89eebae5ebcae61 P 988811a677384142bc818cc6fdbe300a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 988811a677384142bc818cc6fdbe300a, d8a00218b70349268f66766b52fca693; no voters:
I20260811 19:04:14.627419 2567 raft_consensus.cc:2454] T 9dc80b0c541d4a30a89eebae5ebcae61 P 97715af0378444829a4d367bcd618e94 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 988811a677384142bc818cc6fdbe300a in term 1.
I20260811 19:04:14.627612 2944 raft_consensus.cc:2781] T 9dc80b0c541d4a30a89eebae5ebcae61 P 988811a677384142bc818cc6fdbe300a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:04:14.627637 2930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:14.623332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.3:37311 (local address 127.13.109.1:39915)
0811 19:04:14.623572 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:14.623576 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:14.623589 (+ 13us) server_negotiation.cc:407] Connection header received
0811 19:04:14.623629 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:14.623631 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:14.623670 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:14.623743 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:04:14.624014 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:14.624600 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:14.626065 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:14.626631 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:14.626759 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:14.626814 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:14.627444 (+ 630us) server_negotiation.cc:299] Negotiation successful
0811 19:04:14.627518 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148}
I20260811 19:04:14.627780 2934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:14.623232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.3:58813 (local address 127.13.109.2:39669)
0811 19:04:14.623888 (+ 656us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:14.623894 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:14.623907 (+ 13us) server_negotiation.cc:407] Connection header received
0811 19:04:14.623947 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:14.623949 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:14.623989 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:14.624055 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:04:14.624333 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:14.624961 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:14.626442 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:14.627046 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:14.627159 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:14.627213 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:14.627589 (+ 376us) server_negotiation.cc:299] Negotiation successful
0811 19:04:14.627672 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":584}
I20260811 19:04:14.627853 2944 raft_consensus.cc:686] T 9dc80b0c541d4a30a89eebae5ebcae61 P 988811a677384142bc818cc6fdbe300a [term 1 LEADER]: Becoming Leader. State: Replica: 988811a677384142bc818cc6fdbe300a, State: Running, Role: LEADER
I20260811 19:04:14.628153 2950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:14.622984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.2:39669 (local address 127.13.109.3:58813)
0811 19:04:14.623274 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:14.623288 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 19:04:14.623386 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:04:14.624073 (+ 687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:04:14.624076 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:04:14.624094 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:04:14.624204 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:14.624210 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:14.625107 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:14.625111 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:14.626288 (+ 1177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:14.626299 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:14.627323 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:14.627328 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:14.627434 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:14.627464 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 19:04:14.627947 (+ 483us) client_negotiation.cc:240] Negotiation successful
0811 19:04:14.628020 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":87,"threads_started":1}
I20260811 19:04:14.628136 2567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dc80b0c541d4a30a89eebae5ebcae61" candidate_uuid: "d8a00218b70349268f66766b52fca693" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97715af0378444829a4d367bcd618e94" is_pre_election: true
I20260811 19:04:14.628225 2944 consensus_queue.cc:237] T 9dc80b0c541d4a30a89eebae5ebcae61 P 988811a677384142bc818cc6fdbe300a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97715af0378444829a4d367bcd618e94" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 39915 } } peers { permanent_uuid: "988811a677384142bc818cc6fdbe300a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 39669 } } peers { permanent_uuid: "d8a00218b70349268f66766b52fca693" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 42495 } }
I20260811 19:04:14.628330 2567 raft_consensus.cc:2379] T 9dc80b0c541d4a30a89eebae5ebcae61 P 97715af0378444829a4d367bcd618e94 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d8a00218b70349268f66766b52fca693 in current term 1: Already voted for candidate 988811a677384142bc818cc6fdbe300a in this term.
I20260811 19:04:14.628959 2712 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dc80b0c541d4a30a89eebae5ebcae61" candidate_uuid: "d8a00218b70349268f66766b52fca693" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "988811a677384142bc818cc6fdbe300a" is_pre_election: true
I20260811 19:04:14.629573 2790 leader_election.cc:302] T 9dc80b0c541d4a30a89eebae5ebcae61 P d8a00218b70349268f66766b52fca693 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d8a00218b70349268f66766b52fca693; no voters: 97715af0378444829a4d367bcd618e94, 988811a677384142bc818cc6fdbe300a
I20260811 19:04:14.629850 2946 raft_consensus.cc:2726] T 9dc80b0c541d4a30a89eebae5ebcae61 P d8a00218b70349268f66766b52fca693 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 19:04:14.631714 2406 catalog_manager.cc:5495] T 9dc80b0c541d4a30a89eebae5ebcae61 P 988811a677384142bc818cc6fdbe300a reported cstate change: term changed from 0 to 1, leader changed from <none> to 988811a677384142bc818cc6fdbe300a (127.13.109.2). New cstate: current_term: 1 leader_uuid: "988811a677384142bc818cc6fdbe300a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97715af0378444829a4d367bcd618e94" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 39915 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "988811a677384142bc818cc6fdbe300a" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 39669 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d8a00218b70349268f66766b52fca693" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 42495 } health_report { overall_health: UNKNOWN } } }
I20260811 19:04:14.666306 2934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:14.662532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44726 (local address 127.13.109.2:39669)
0811 19:04:14.662694 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:14.662699 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:14.662714 (+ 15us) server_negotiation.cc:407] Connection header received
0811 19:04:14.662887 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:14.662890 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:14.662947 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:14.663088 (+ 141us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:04:14.663357 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:14.663860 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:14.664745 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:14.664969 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:14.665098 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:14.665434 (+ 336us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:04:14.665908 (+ 474us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:04:14.666053 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:14.666099 (+ 46us) server_negotiation.cc:299] Negotiation successful
0811 19:04:14.666155 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 19:04:14.672820 2567 raft_consensus.cc:1261] T 9dc80b0c541d4a30a89eebae5ebcae61 P 97715af0378444829a4d367bcd618e94 [term 1 FOLLOWER]: Refusing update from remote peer 988811a677384142bc818cc6fdbe300a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:04:14.672930 2865 raft_consensus.cc:1261] T 9dc80b0c541d4a30a89eebae5ebcae61 P d8a00218b70349268f66766b52fca693 [term 1 FOLLOWER]: Refusing update from remote peer 988811a677384142bc818cc6fdbe300a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:04:14.673533 2951 consensus_queue.cc:1035] T 9dc80b0c541d4a30a89eebae5ebcae61 P 988811a677384142bc818cc6fdbe300a [LEADER]: Connected to new peer: Peer: permanent_uuid: "97715af0378444829a4d367bcd618e94" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 39915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:04:14.673732 2944 consensus_queue.cc:1035] T 9dc80b0c541d4a30a89eebae5ebcae61 P 988811a677384142bc818cc6fdbe300a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8a00218b70349268f66766b52fca693" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 42495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:04:14.687143 2956 mvcc.cc:205] Tried to move back new op lower bound from 7317401823931973632 to 7317401823760359424. Current Snapshot: MvccSnapshot[applied={T|T < 7317401823931973632}]
I20260811 19:04:14.691100 2406 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34696:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:04:14.691304 2406 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34696:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:04:14.694023 2406 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 193ff6c6619a4ee78295631a2293cb13: Sending DeleteTablet for 3 replicas of tablet 9dc80b0c541d4a30a89eebae5ebcae61
I20260811 19:04:14.695005 2543 tablet_service.cc:1514] Processing DeleteTablet for tablet 9dc80b0c541d4a30a89eebae5ebcae61 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:04:14 UTC) from {username='kudu', principal='kudu/127.13.109.62@KRBTEST.COM'} at 127.0.0.1:38656
I20260811 19:04:14.695127 2691 tablet_service.cc:1514] Processing DeleteTablet for tablet 9dc80b0c541d4a30a89eebae5ebcae61 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:04:14 UTC) from {username='kudu', principal='kudu/127.13.109.62@KRBTEST.COM'} at 127.0.0.1:44722
I20260811 19:04:14.695143 2843 tablet_service.cc:1514] Processing DeleteTablet for tablet 9dc80b0c541d4a30a89eebae5ebcae61 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:04:14 UTC) from {username='kudu', principal='kudu/127.13.109.62@KRBTEST.COM'} at 127.0.0.1:55076
I20260811 19:04:14.695628 2964 tablet_replica.cc:330] T 9dc80b0c541d4a30a89eebae5ebcae61 P 988811a677384142bc818cc6fdbe300a: stopping tablet replica
I20260811 19:04:14.695793 2967 tablet_replica.cc:330] T 9dc80b0c541d4a30a89eebae5ebcae61 P d8a00218b70349268f66766b52fca693: stopping tablet replica
I20260811 19:04:14.696000 2964 raft_consensus.cc:2229] T 9dc80b0c541d4a30a89eebae5ebcae61 P 988811a677384142bc818cc6fdbe300a [term 1 LEADER]: Raft consensus shutting down.
I20260811 19:04:14.696089 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 2476
I20260811 19:04:14.696206 2967 raft_consensus.cc:2229] T 9dc80b0c541d4a30a89eebae5ebcae61 P d8a00218b70349268f66766b52fca693 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:04:14.696408 2964 raft_consensus.cc:2258] T 9dc80b0c541d4a30a89eebae5ebcae61 P 988811a677384142bc818cc6fdbe300a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:04:14.696797 2967 raft_consensus.cc:2258] T 9dc80b0c541d4a30a89eebae5ebcae61 P d8a00218b70349268f66766b52fca693 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:04:14.698688 2967 ts_tablet_manager.cc:1905] T 9dc80b0c541d4a30a89eebae5ebcae61 P d8a00218b70349268f66766b52fca693: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:04:14.700806 2964 ts_tablet_manager.cc:1905] T 9dc80b0c541d4a30a89eebae5ebcae61 P 988811a677384142bc818cc6fdbe300a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:04:14.702332 2967 ts_tablet_manager.cc:1918] T 9dc80b0c541d4a30a89eebae5ebcae61 P d8a00218b70349268f66766b52fca693: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:04:14.702524 2967 log.cc:1196] T 9dc80b0c541d4a30a89eebae5ebcae61 P d8a00218b70349268f66766b52fca693: Deleting WAL directory at /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/minicluster-data/ts-2/wal/wals/9dc80b0c541d4a30a89eebae5ebcae61
I20260811 19:04:14.702953 2967 ts_tablet_manager.cc:1939] T 9dc80b0c541d4a30a89eebae5ebcae61 P d8a00218b70349268f66766b52fca693: Deleting consensus metadata
I20260811 19:04:14.703800 2964 ts_tablet_manager.cc:1918] T 9dc80b0c541d4a30a89eebae5ebcae61 P 988811a677384142bc818cc6fdbe300a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:04:14.703915 2964 log.cc:1196] T 9dc80b0c541d4a30a89eebae5ebcae61 P 988811a677384142bc818cc6fdbe300a: Deleting WAL directory at /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474792496492-13748-0/minicluster-data/ts-1/wal/wals/9dc80b0c541d4a30a89eebae5ebcae61
I20260811 19:04:14.704243 2964 ts_tablet_manager.cc:1939] T 9dc80b0c541d4a30a89eebae5ebcae61 P 988811a677384142bc818cc6fdbe300a: Deleting consensus metadata
I20260811 19:04:14.704855 2392 catalog_manager.cc:4841] TS d8a00218b70349268f66766b52fca693 (127.13.109.3:42495): tablet 9dc80b0c541d4a30a89eebae5ebcae61 (table test-table [id=39665d33f22e4bfc998ca024ff5454f7]) successfully deleted
I20260811 19:04:14.705713 2391 catalog_manager.cc:4841] TS 988811a677384142bc818cc6fdbe300a (127.13.109.2:39669): tablet 9dc80b0c541d4a30a89eebae5ebcae61 (table test-table [id=39665d33f22e4bfc998ca024ff5454f7]) successfully deleted
W20260811 19:04:14.707294 2391 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.109.1:39915 (error 108)
I20260811 19:04:14.707360 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 2623
I20260811 19:04:14.710099 2969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:14.709656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.1:39915 (local address 127.0.0.1:38658)
0811 19:04:14.709902 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:14.709989 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.1:39915: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":87,"threads_started":1}
W20260811 19:04:14.710325 2391 catalog_manager.cc:4584] TS 97715af0378444829a4d367bcd618e94 (127.13.109.1:39915): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9dc80b0c541d4a30a89eebae5ebcae61: Network error: Client connection negotiation failed: client connection to 127.13.109.1:39915: connect: Connection refused (error 111)
I20260811 19:04:14.715764 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 2773
I20260811 19:04:14.722942 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 2372
2026-08-11T19:04:14Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2806 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-11T19:04:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:04:14Z Disabled control of system clock
I20260811 19:04:14.757283 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474792496492-13748-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474792496492-13748-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474792496492-13748-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.62:35469
--webserver_interface=127.13.109.62
--webserver_port=0
--builtin_ntp_servers=127.13.109.20:33101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.109.62:35469
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 19:04:14.877050 2973 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:04:14.877455 2973 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:04:14.877547 2973 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:04:14.882319 2973 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:04:14.882444 2973 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:04:14.882472 2973 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:04:14.882505 2973 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:04:14.882565 2973 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:04:14.888031 2973 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:33101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474792496492-13748-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474792496492-13748-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.109.62:35469
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.62:35469
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.109.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2973
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474792496492-13748-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:04:14.889379 2973 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:04:14.891016 2973 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:04:14.900441 2978 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:04:14.900434 2981 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:04:14.900426 2979 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:04:14.900658 2973 server_base.cc:1034] running on GCE node
I20260811 19:04:14.901536 2973 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:04:14.902839 2973 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:04:14.904071 2973 hybrid_clock.cc:648] HybridClock initialized: now 1786475054904002 us; error 50 us; skew 500 ppm
I20260811 19:04:14.909670 2973 webserver.cc:460] Webserver started at http://127.13.109.62:40837/ using document root <none> and password file <none>
I20260811 19:04:14.910446 2973 fs_manager.cc:359] Metadata directory not provided
I20260811 19:04:14.910531 2973 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:04:14.910720 2973 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:04:14.913003 2973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474792496492-13748-0/minicluster-data/master-0/data/instance:
uuid: "6d39ae7810d8493a8cbb04aaca8ec900"
format_stamp: "Formatted at 2026-08-11 19:04:14 on dist-test-slave-0kls"
server_key: "957879fe9721799848e5022944708174"
server_key_iv: "59ebe64989b71e1f870a0523a533614d"
server_key_version: "encryptionkey@0"
I20260811 19:04:14.913594 2973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474792496492-13748-0/minicluster-data/master-0/wal/instance:
uuid: "6d39ae7810d8493a8cbb04aaca8ec900"
format_stamp: "Formatted at 2026-08-11 19:04:14 on dist-test-slave-0kls"
server_key: "957879fe9721799848e5022944708174"
server_key_iv: "59ebe64989b71e1f870a0523a533614d"
server_key_version: "encryptionkey@0"
I20260811 19:04:14.917964 2973 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 19:04:14.920881 2987 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:04:14.922506 2973 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 19:04:14.922688 2973 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474792496492-13748-0/minicluster-data/master-0/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474792496492-13748-0/minicluster-data/master-0/wal
uuid: "6d39ae7810d8493a8cbb04aaca8ec900"
format_stamp: "Formatted at 2026-08-11 19:04:14 on dist-test-slave-0kls"
server_key: "957879fe9721799848e5022944708174"
server_key_iv: "59ebe64989b71e1f870a0523a533614d"
server_key_version: "encryptionkey@0"
I20260811 19:04:14.922832 2973 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474792496492-13748-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474792496492-13748-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474792496492-13748-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:04:14.941718 2973 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:04:14.942672 2973 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:04:14.942905 2973 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:04:14.951673 2973 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.62:35469
I20260811 19:04:14.951679 3047 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.62:35469 every 8 connection(s)
I20260811 19:04:14.952792 2973 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
I20260811 19:04:14.954499 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 2973
I20260811 19:04:14.954697 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474792496492-13748-0/minicluster-data/master-0/wal/instance
I20260811 19:04:14.955024 13748 external_mini_cluster.cc:1428] Setting key bf5253d4bd0b53b262cf28036e5aab5e
I20260811 19:04:14.956882 3048 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:04:14.963258 3048 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d39ae7810d8493a8cbb04aaca8ec900: Bootstrap starting.
I20260811 19:04:14.965329 3052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:14.956618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38600 (local address 127.13.109.62:35469)
0811 19:04:14.957429 (+ 811us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:14.957446 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:14.957489 (+ 43us) server_negotiation.cc:407] Connection header received
0811 19:04:14.958281 (+ 792us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:14.958302 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:14.958736 (+ 434us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:14.959107 (+ 371us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:14.959686 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:14.960749 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:14.961693 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:14.962274 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:14.962460 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:14.963473 (+ 1013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:04:14.963482 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:04:14.963500 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:04:14.963943 (+ 443us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:04:14.963999 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:04:14.964021 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:04:14.964149 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:14.964273 (+ 124us) server_negotiation.cc:299] Negotiation successful
0811 19:04:14.964562 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":419,"thread_start_us":184,"threads_started":1}
I20260811 19:04:14.966069 3048 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6d39ae7810d8493a8cbb04aaca8ec900: Neither blocks nor log segments found. Creating new log.
I20260811 19:04:14.966959 3048 log.cc:824] T 00000000000000000000000000000000 P 6d39ae7810d8493a8cbb04aaca8ec900: Log is configured to *not* fsync() on all Append() calls
I20260811 19:04:14.969252 3048 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d39ae7810d8493a8cbb04aaca8ec900: No bootstrap required, opened a new log
I20260811 19:04:14.973220 3048 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6d39ae7810d8493a8cbb04aaca8ec900 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d39ae7810d8493a8cbb04aaca8ec900" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 35469 } }
I20260811 19:04:14.973559 3048 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6d39ae7810d8493a8cbb04aaca8ec900 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:04:14.973634 3048 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6d39ae7810d8493a8cbb04aaca8ec900 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d39ae7810d8493a8cbb04aaca8ec900, State: Initialized, Role: FOLLOWER
I20260811 19:04:14.974186 3048 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6d39ae7810d8493a8cbb04aaca8ec900 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d39ae7810d8493a8cbb04aaca8ec900" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 35469 } }
I20260811 19:04:14.974323 3048 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6d39ae7810d8493a8cbb04aaca8ec900 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:04:14.974411 3048 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6d39ae7810d8493a8cbb04aaca8ec900 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:04:14.974519 3048 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6d39ae7810d8493a8cbb04aaca8ec900 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:04:14.975553 3048 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6d39ae7810d8493a8cbb04aaca8ec900 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d39ae7810d8493a8cbb04aaca8ec900" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 35469 } }
I20260811 19:04:14.975922 3048 leader_election.cc:302] T 00000000000000000000000000000000 P 6d39ae7810d8493a8cbb04aaca8ec900 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6d39ae7810d8493a8cbb04aaca8ec900; no voters:
I20260811 19:04:14.976279 3048 leader_election.cc:288] T 00000000000000000000000000000000 P 6d39ae7810d8493a8cbb04aaca8ec900 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:04:14.976483 3054 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6d39ae7810d8493a8cbb04aaca8ec900 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:04:14.976751 3054 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6d39ae7810d8493a8cbb04aaca8ec900 [term 1 LEADER]: Becoming Leader. State: Replica: 6d39ae7810d8493a8cbb04aaca8ec900, State: Running, Role: LEADER
I20260811 19:04:14.977105 3054 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6d39ae7810d8493a8cbb04aaca8ec900 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d39ae7810d8493a8cbb04aaca8ec900" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 35469 } }
I20260811 19:04:14.977706 3048 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6d39ae7810d8493a8cbb04aaca8ec900 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:04:14.978921 3055 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d39ae7810d8493a8cbb04aaca8ec900 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6d39ae7810d8493a8cbb04aaca8ec900" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d39ae7810d8493a8cbb04aaca8ec900" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 35469 } } }
I20260811 19:04:14.979082 3055 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d39ae7810d8493a8cbb04aaca8ec900 [sys.catalog]: This master's current role is: LEADER
I20260811 19:04:14.979413 3056 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d39ae7810d8493a8cbb04aaca8ec900 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6d39ae7810d8493a8cbb04aaca8ec900. Latest consensus state: current_term: 1 leader_uuid: "6d39ae7810d8493a8cbb04aaca8ec900" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d39ae7810d8493a8cbb04aaca8ec900" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 35469 } } }
I20260811 19:04:14.979511 3056 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d39ae7810d8493a8cbb04aaca8ec900 [sys.catalog]: This master's current role is: LEADER
I20260811 19:04:14.979653 3063 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:04:14.982873 3063 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:04:14.989836 3063 catalog_manager.cc:1347] Generated new cluster ID: ba79ee36b96b4c238564102c693e3dbc
I20260811 19:04:14.989944 3063 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:04:14.995262 3063 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:04:14.996616 3063 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:04:15.006683 3063 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6d39ae7810d8493a8cbb04aaca8ec900: Generated new TSK 0
I20260811 19:04:15.007505 3063 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 19:04:15.017640 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474792496492-13748-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474792496492-13748-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474792496492-13748-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.1:0
--local_ip_for_outbound_sockets=127.13.109.1
--webserver_interface=127.13.109.1
--webserver_port=0
--tserver_master_addrs=127.13.109.62:35469
--builtin_ntp_servers=127.13.109.20:33101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 19:04:15.147117 3074 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:04:15.147506 3074 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:04:15.147583 3074 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:04:15.152380 3074 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:04:15.152524 3074 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:04:15.152644 3074 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.1
I20260811 19:04:15.158309 3074 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:33101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474792496492-13748-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474792496492-13748-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.109.1
--webserver_port=0
--tserver_master_addrs=127.13.109.62:35469
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3074
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.1
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474792496492-13748-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:04:15.159657 3074 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:04:15.161247 3074 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:04:15.171887 3082 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:04:15.172055 3074 server_base.cc:1034] running on GCE node
W20260811 19:04:15.171890 3080 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:04:15.171893 3079 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:04:15.172761 3074 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:04:15.173553 3074 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:04:15.174788 3074 hybrid_clock.cc:648] HybridClock initialized: now 1786475055174766 us; error 41 us; skew 500 ppm
I20260811 19:04:15.176942 3074 webserver.cc:460] Webserver started at http://127.13.109.1:37979/ using document root <none> and password file <none>
I20260811 19:04:15.177618 3074 fs_manager.cc:359] Metadata directory not provided
I20260811 19:04:15.177723 3074 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:04:15.177999 3074 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:04:15.180938 3074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474792496492-13748-0/minicluster-data/ts-0/data/instance:
uuid: "92e98b8bd63342249f98111ff0f94b87"
format_stamp: "Formatted at 2026-08-11 19:04:15 on dist-test-slave-0kls"
server_key: "80d0715933a341b20d1692d4b3eb25af"
server_key_iv: "1ee6ef9090bb1912b99c6f77cf7d030f"
server_key_version: "encryptionkey@0"
I20260811 19:04:15.181665 3074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474792496492-13748-0/minicluster-data/ts-0/wal/instance:
uuid: "92e98b8bd63342249f98111ff0f94b87"
format_stamp: "Formatted at 2026-08-11 19:04:15 on dist-test-slave-0kls"
server_key: "80d0715933a341b20d1692d4b3eb25af"
server_key_iv: "1ee6ef9090bb1912b99c6f77cf7d030f"
server_key_version: "encryptionkey@0"
I20260811 19:04:15.186210 3074 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 19:04:15.189247 3088 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:04:15.190783 3074 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 19:04:15.190964 3074 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474792496492-13748-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474792496492-13748-0/minicluster-data/ts-0/wal
uuid: "92e98b8bd63342249f98111ff0f94b87"
format_stamp: "Formatted at 2026-08-11 19:04:15 on dist-test-slave-0kls"
server_key: "80d0715933a341b20d1692d4b3eb25af"
server_key_iv: "1ee6ef9090bb1912b99c6f77cf7d030f"
server_key_version: "encryptionkey@0"
I20260811 19:04:15.191154 3074 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474792496492-13748-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474792496492-13748-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474792496492-13748-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:04:15.213827 3074 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:04:15.214789 3074 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:04:15.215063 3074 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:04:15.216117 3074 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:04:15.217324 3074 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:04:15.217413 3074 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:04:15.217495 3074 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:04:15.217542 3074 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:04:15.227993 3074 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.1:35509
I20260811 19:04:15.228006 3213 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.1:35509 every 8 connection(s)
I20260811 19:04:15.229076 3074 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474792496492-13748-0/minicluster-data/ts-0/data/info.pb
I20260811 19:04:15.237110 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 3074
I20260811 19:04:15.237262 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474792496492-13748-0/minicluster-data/ts-0/wal/instance
I20260811 19:04:15.237603 13748 external_mini_cluster.cc:1428] Setting key aafa5b7319896b98273cb8fe99c10f85
I20260811 19:04:15.240274 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474792496492-13748-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474792496492-13748-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474792496492-13748-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.2:0
--local_ip_for_outbound_sockets=127.13.109.2
--webserver_interface=127.13.109.2
--webserver_port=0
--tserver_master_addrs=127.13.109.62:35469
--builtin_ntp_servers=127.13.109.20:33101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 19:04:15.240726 3052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:15.231398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.1:33861 (local address 127.13.109.62:35469)
0811 19:04:15.231560 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:15.231565 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:15.232899 (+ 1334us) server_negotiation.cc:407] Connection header received
0811 19:04:15.234420 (+ 1521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:15.234424 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:15.234488 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:15.234582 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:15.235589 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:15.236215 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:15.237885 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:15.238179 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:15.238331 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:15.240039 (+ 1708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:04:15.240042 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:04:15.240044 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:04:15.240059 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:04:15.240085 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:04:15.240093 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:04:15.240165 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:15.240454 (+ 289us) server_negotiation.cc:299] Negotiation successful
0811 19:04:15.240557 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 19:04:15.242053 3216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:15.231916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:35469 (local address 127.13.109.1:33861)
0811 19:04:15.232659 (+ 743us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:15.232705 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 19:04:15.234177 (+ 1472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:04:15.234766 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:04:15.234773 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:04:15.234866 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:04:15.235436 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:15.235450 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:15.236361 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:15.236364 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:15.237705 (+ 1341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:15.237719 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:15.238403 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:15.238407 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:15.238729 (+ 322us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:15.239641 (+ 912us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 19:04:15.239643 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:04:15.239710 (+ 67us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 19:04:15.239828 (+ 118us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:04:15.240197 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:04:15.240208 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:04:15.240216 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 19:04:15.240381 (+ 165us) client_negotiation.cc:240] Negotiation successful
0811 19:04:15.240769 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":127,"threads_started":1}
I20260811 19:04:15.243539 3214 heartbeater.cc:347] Connected to a master server at 127.13.109.62:35469
I20260811 19:04:15.243830 3214 heartbeater.cc:464] Registering TS with master...
I20260811 19:04:15.244400 3214 heartbeater.cc:511] Master 127.13.109.62:35469 requested a full tablet report, sending...
I20260811 19:04:15.246145 3006 ts_manager.cc:194] Registered new tserver with Master: 92e98b8bd63342249f98111ff0f94b87 (127.13.109.1:35509)
I20260811 19:04:15.247412 3006 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.109.1:33861
W20260811 19:04:15.365657 3217 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:04:15.366046 3217 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:04:15.366124 3217 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:04:15.370422 3217 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:04:15.370517 3217 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:04:15.370620 3217 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.2
I20260811 19:04:15.375661 3217 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:33101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474792496492-13748-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474792496492-13748-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.109.2
--webserver_port=0
--tserver_master_addrs=127.13.109.62:35469
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3217
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.2
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474792496492-13748-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:04:15.376929 3217 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:04:15.378616 3217 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:04:15.389372 3225 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:04:15.389353 3222 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:04:15.389372 3223 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:04:15.389693 3217 server_base.cc:1034] running on GCE node
I20260811 19:04:15.390205 3217 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:04:15.390976 3217 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:04:15.392195 3217 hybrid_clock.cc:648] HybridClock initialized: now 1786475055392157 us; error 49 us; skew 500 ppm
I20260811 19:04:15.394373 3217 webserver.cc:460] Webserver started at http://127.13.109.2:38513/ using document root <none> and password file <none>
I20260811 19:04:15.395071 3217 fs_manager.cc:359] Metadata directory not provided
I20260811 19:04:15.395200 3217 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:04:15.395447 3217 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:04:15.397758 3217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474792496492-13748-0/minicluster-data/ts-1/data/instance:
uuid: "bc6ff4e413564c05bd52b2f966c12274"
format_stamp: "Formatted at 2026-08-11 19:04:15 on dist-test-slave-0kls"
server_key: "588c109cfb90df2f95f31326ba60644d"
server_key_iv: "586a568db72950f467fbbb2b98b68357"
server_key_version: "encryptionkey@0"
I20260811 19:04:15.398439 3217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474792496492-13748-0/minicluster-data/ts-1/wal/instance:
uuid: "bc6ff4e413564c05bd52b2f966c12274"
format_stamp: "Formatted at 2026-08-11 19:04:15 on dist-test-slave-0kls"
server_key: "588c109cfb90df2f95f31326ba60644d"
server_key_iv: "586a568db72950f467fbbb2b98b68357"
server_key_version: "encryptionkey@0"
I20260811 19:04:15.402853 3217 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 19:04:15.405879 3233 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:04:15.407420 3217 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 19:04:15.407585 3217 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474792496492-13748-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474792496492-13748-0/minicluster-data/ts-1/wal
uuid: "bc6ff4e413564c05bd52b2f966c12274"
format_stamp: "Formatted at 2026-08-11 19:04:15 on dist-test-slave-0kls"
server_key: "588c109cfb90df2f95f31326ba60644d"
server_key_iv: "586a568db72950f467fbbb2b98b68357"
server_key_version: "encryptionkey@0"
I20260811 19:04:15.407732 3217 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474792496492-13748-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474792496492-13748-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474792496492-13748-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:04:15.417987 3217 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:04:15.418798 3217 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:04:15.419025 3217 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:04:15.419698 3217 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:04:15.420845 3217 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:04:15.420934 3217 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:04:15.421028 3217 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:04:15.421082 3217 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:04:15.431641 3217 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.2:43367
I20260811 19:04:15.431674 3356 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.2:43367 every 8 connection(s)
I20260811 19:04:15.432848 3217 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474792496492-13748-0/minicluster-data/ts-1/data/info.pb
I20260811 19:04:15.440559 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 3217
I20260811 19:04:15.440742 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474792496492-13748-0/minicluster-data/ts-1/wal/instance
I20260811 19:04:15.441084 13748 external_mini_cluster.cc:1428] Setting key 72a63ab6d1baf505bfd9390c904a4e67
I20260811 19:04:15.443188 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474792496492-13748-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474792496492-13748-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474792496492-13748-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.109.3:0
--local_ip_for_outbound_sockets=127.13.109.3
--webserver_interface=127.13.109.3
--webserver_port=0
--tserver_master_addrs=127.13.109.62:35469
--builtin_ntp_servers=127.13.109.20:33101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 19:04:15.443662 3052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:15.434985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.2:51869 (local address 127.13.109.62:35469)
0811 19:04:15.435141 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:15.435145 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:15.436636 (+ 1491us) server_negotiation.cc:407] Connection header received
0811 19:04:15.438243 (+ 1607us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:15.438247 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:15.438318 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:15.438426 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:15.439458 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:15.440022 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:15.441453 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:15.441688 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:15.441847 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:15.443087 (+ 1240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:04:15.443090 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:04:15.443095 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:04:15.443114 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:04:15.443137 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:04:15.443145 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:04:15.443226 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:15.443445 (+ 219us) server_negotiation.cc:299] Negotiation successful
0811 19:04:15.443516 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 19:04:15.444653 3361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:15.435669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:35469 (local address 127.13.109.2:51869)
0811 19:04:15.436427 (+ 758us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:15.436473 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 19:04:15.437987 (+ 1514us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:04:15.438637 (+ 650us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:04:15.438645 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:04:15.438744 (+ 99us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:04:15.439291 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:15.439303 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:15.440174 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:15.440183 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:15.441313 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:15.441325 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:15.441911 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:15.441916 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:15.442100 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:15.442783 (+ 683us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 19:04:15.442785 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:04:15.442832 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 19:04:15.442909 (+ 77us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:04:15.443229 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:04:15.443241 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:04:15.443251 (+ 10us) client_negotiation.cc:769] Sending connection context
0811 19:04:15.443438 (+ 187us) client_negotiation.cc:240] Negotiation successful
0811 19:04:15.443781 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":130,"threads_started":1}
I20260811 19:04:15.445940 3358 heartbeater.cc:347] Connected to a master server at 127.13.109.62:35469
I20260811 19:04:15.446231 3358 heartbeater.cc:464] Registering TS with master...
I20260811 19:04:15.446846 3358 heartbeater.cc:511] Master 127.13.109.62:35469 requested a full tablet report, sending...
I20260811 19:04:15.448067 3006 ts_manager.cc:194] Registered new tserver with Master: bc6ff4e413564c05bd52b2f966c12274 (127.13.109.2:43367)
I20260811 19:04:15.448683 3006 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.109.2:51869
W20260811 19:04:15.565274 3362 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:04:15.565658 3362 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:04:15.565742 3362 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:04:15.569991 3362 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:04:15.570097 3362 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:04:15.570199 3362 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.109.3
I20260811 19:04:15.575220 3362 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:33101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474792496492-13748-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474792496492-13748-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.109.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.109.3
--webserver_port=0
--tserver_master_addrs=127.13.109.62:35469
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3362
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.109.3
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474792496492-13748-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:04:15.576527 3362 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:04:15.578265 3362 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:04:15.589191 3370 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:04:15.589180 3368 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:04:15.589254 3362 server_base.cc:1034] running on GCE node
W20260811 19:04:15.589180 3367 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:04:15.589890 3362 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:04:15.590708 3362 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:04:15.591939 3362 hybrid_clock.cc:648] HybridClock initialized: now 1786475055591923 us; error 39 us; skew 500 ppm
I20260811 19:04:15.594277 3362 webserver.cc:460] Webserver started at http://127.13.109.3:36635/ using document root <none> and password file <none>
I20260811 19:04:15.594976 3362 fs_manager.cc:359] Metadata directory not provided
I20260811 19:04:15.595085 3362 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:04:15.595355 3362 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:04:15.598562 3362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474792496492-13748-0/minicluster-data/ts-2/data/instance:
uuid: "28f3e5299c8d4d1db639b197e62cd43b"
format_stamp: "Formatted at 2026-08-11 19:04:15 on dist-test-slave-0kls"
server_key: "af9c23fabf05a9cf48235a2bc2b914c9"
server_key_iv: "d4a7c6c392f87b2af13049bb3fe4a411"
server_key_version: "encryptionkey@0"
I20260811 19:04:15.599828 3362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474792496492-13748-0/minicluster-data/ts-2/wal/instance:
uuid: "28f3e5299c8d4d1db639b197e62cd43b"
format_stamp: "Formatted at 2026-08-11 19:04:15 on dist-test-slave-0kls"
server_key: "af9c23fabf05a9cf48235a2bc2b914c9"
server_key_iv: "d4a7c6c392f87b2af13049bb3fe4a411"
server_key_version: "encryptionkey@0"
I20260811 19:04:15.604517 3362 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260811 19:04:15.607796 3377 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:04:15.609522 3362 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 19:04:15.609859 3362 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474792496492-13748-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474792496492-13748-0/minicluster-data/ts-2/wal
uuid: "28f3e5299c8d4d1db639b197e62cd43b"
format_stamp: "Formatted at 2026-08-11 19:04:15 on dist-test-slave-0kls"
server_key: "af9c23fabf05a9cf48235a2bc2b914c9"
server_key_iv: "d4a7c6c392f87b2af13049bb3fe4a411"
server_key_version: "encryptionkey@0"
I20260811 19:04:15.610025 3362 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474792496492-13748-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474792496492-13748-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474792496492-13748-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:04:15.622994 3362 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:04:15.623770 3362 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:04:15.623986 3362 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:04:15.624619 3362 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:04:15.625607 3362 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:04:15.625655 3362 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:04:15.625734 3362 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:04:15.625819 3362 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260811 19:04:15.634377 3362 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.3:41667
I20260811 19:04:15.634469 3502 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.3:41667 every 8 connection(s)
I20260811 19:04:15.635349 3362 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474792496492-13748-0/minicluster-data/ts-2/data/info.pb
I20260811 19:04:15.641752 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 3362
I20260811 19:04:15.641932 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474792496492-13748-0/minicluster-data/ts-2/wal/instance
I20260811 19:04:15.642249 13748 external_mini_cluster.cc:1428] Setting key 85b609d0952f83e562097001e8933ee3
I20260811 19:04:15.645131 3052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:15.637206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.3:40269 (local address 127.13.109.62:35469)
0811 19:04:15.637357 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:15.637361 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:15.638413 (+ 1052us) server_negotiation.cc:407] Connection header received
0811 19:04:15.639903 (+ 1490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:15.639907 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:15.639970 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:15.640076 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:15.640987 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:15.641631 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:15.643130 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:15.643463 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:15.643577 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:15.644708 (+ 1131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:04:15.644711 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:04:15.644713 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:04:15.644725 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:04:15.644751 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:04:15.644758 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:04:15.644818 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:15.644968 (+ 150us) server_negotiation.cc:299] Negotiation successful
0811 19:04:15.645019 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 19:04:15.646123 3505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:15.637568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:35469 (local address 127.13.109.3:40269)
0811 19:04:15.638261 (+ 693us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:15.638302 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 19:04:15.639655 (+ 1353us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:04:15.640252 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:04:15.640259 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:04:15.640344 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:04:15.640837 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:15.640849 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:15.641868 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:15.641873 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:15.643010 (+ 1137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:15.643021 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:15.643634 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:15.643637 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:15.643780 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:15.644457 (+ 677us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 19:04:15.644459 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:04:15.644501 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 19:04:15.644572 (+ 71us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:04:15.644845 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:04:15.644852 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:04:15.644857 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 19:04:15.644953 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 19:04:15.645213 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":171,"threads_started":1}
I20260811 19:04:15.647330 3503 heartbeater.cc:347] Connected to a master server at 127.13.109.62:35469
I20260811 19:04:15.647547 3503 heartbeater.cc:464] Registering TS with master...
I20260811 19:04:15.648101 3503 heartbeater.cc:511] Master 127.13.109.62:35469 requested a full tablet report, sending...
I20260811 19:04:15.649143 3006 ts_manager.cc:194] Registered new tserver with Master: 28f3e5299c8d4d1db639b197e62cd43b (127.13.109.3:41667)
I20260811 19:04:15.649695 3006 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.109.3:40269
I20260811 19:04:15.656396 13748 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 19:04:15.662705 3052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:15.658559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38614 (local address 127.13.109.62:35469)
0811 19:04:15.658728 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:15.658733 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:15.658876 (+ 143us) server_negotiation.cc:407] Connection header received
0811 19:04:15.659052 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:15.659058 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:15.659116 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:15.659205 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:15.659512 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:15.660072 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:15.660940 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:15.661179 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:15.661298 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:15.662219 (+ 921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:04:15.662222 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:04:15.662224 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:04:15.662238 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:04:15.662259 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:04:15.662267 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:04:15.662346 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:15.662473 (+ 127us) server_negotiation.cc:299] Negotiation successful
0811 19:04:15.662545 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 19:04:15.665937 3006 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38614:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 19:04:15.669099 3006 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 19:04:15.686038 3516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:15.679179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52312 (local address 127.13.109.1:35509)
0811 19:04:15.681453 (+ 2274us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:15.681471 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:15.681492 (+ 21us) server_negotiation.cc:407] Connection header received
0811 19:04:15.681552 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:15.681556 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:15.681686 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:15.681850 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:04:15.682094 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:15.683154 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:15.684382 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:15.685148 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:15.685294 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:15.685526 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:15.685724 (+ 198us) server_negotiation.cc:299] Negotiation successful
0811 19:04:15.685859 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2148,"thread_start_us":108,"threads_started":1}
I20260811 19:04:15.686033 3517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:15.679181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.3:41667 (local address 127.0.0.1:43762)
0811 19:04:15.680144 (+ 963us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:15.680553 (+ 409us) client_negotiation.cc:174] Beginning negotiation
0811 19:04:15.680636 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:04:15.681132 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:04:15.681138 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:04:15.681160 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:04:15.681320 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:15.681328 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:15.682543 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:15.682546 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:15.684043 (+ 1497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:15.684055 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:15.685325 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:15.685329 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:15.685486 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:15.685535 (+ 49us) client_negotiation.cc:769] Sending connection context
0811 19:04:15.685671 (+ 136us) client_negotiation.cc:240] Negotiation successful
0811 19:04:15.685855 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":863,"mutex_wait_us":54,"thread_start_us":126,"threads_started":1}
I20260811 19:04:15.686311 3514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:15.679070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.1:35509 (local address 127.0.0.1:52312)
0811 19:04:15.680144 (+ 1074us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:15.680204 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 19:04:15.680352 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:04:15.681845 (+ 1493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:04:15.681848 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:04:15.681867 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:04:15.681964 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:15.681970 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:15.683307 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:15.683311 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:15.684238 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:15.684248 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:15.685287 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:15.685291 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:15.685467 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:15.685536 (+ 69us) client_negotiation.cc:769] Sending connection context
0811 19:04:15.685660 (+ 124us) client_negotiation.cc:240] Negotiation successful
0811 19:04:15.685863 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":889,"mutex_wait_us":70,"thread_start_us":115,"threads_started":1}
I20260811 19:04:15.686661 3519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:15.680062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.2:43367 (local address 127.0.0.1:58312)
0811 19:04:15.680577 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:15.680594 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 19:04:15.680699 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:04:15.682512 (+ 1813us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:04:15.682518 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:04:15.682543 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:04:15.682676 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:15.682683 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:15.684110 (+ 1427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:15.684114 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:15.684921 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:15.684931 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:15.686255 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:15.686258 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:15.686351 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:15.686381 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 19:04:15.686458 (+ 77us) client_negotiation.cc:240] Negotiation successful
0811 19:04:15.686548 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":442,"mutex_wait_us":21,"thread_start_us":106,"threads_started":1}
I20260811 19:04:15.687044 3520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:15.680504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58312 (local address 127.13.109.2:43367)
0811 19:04:15.682064 (+ 1560us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:15.682078 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:15.682100 (+ 22us) server_negotiation.cc:407] Connection header received
0811 19:04:15.682177 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:15.682182 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:15.682355 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:15.682510 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:04:15.682814 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:15.683972 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:15.685060 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:15.685888 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:15.685993 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:15.686178 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:15.686822 (+ 644us) server_negotiation.cc:299] Negotiation successful
0811 19:04:15.686920 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1423,"thread_start_us":114,"threads_started":1}
I20260811 19:04:15.687819 3518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:15.679353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43762 (local address 127.13.109.3:41667)
0811 19:04:15.679769 (+ 416us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:15.679800 (+ 31us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:15.680716 (+ 916us) server_negotiation.cc:407] Connection header received
0811 19:04:15.680802 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:15.680806 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:15.680972 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:15.681143 (+ 171us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:04:15.681447 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:15.682404 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:15.684281 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:15.685148 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:15.687409 (+ 2261us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:15.687580 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:15.687621 (+ 41us) server_negotiation.cc:299] Negotiation successful
0811 19:04:15.687696 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":247,"thread_start_us":129,"threads_started":1}
I20260811 19:04:15.690284 3141 tablet_service.cc:1467] Processing CreateTablet for tablet d46085a06902451c8a9ff416a4e5d713 (DEFAULT_TABLE table=test-table [id=696bc160a78b4091bd8f9e7078a5357a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:04:15.690310 3286 tablet_service.cc:1467] Processing CreateTablet for tablet d46085a06902451c8a9ff416a4e5d713 (DEFAULT_TABLE table=test-table [id=696bc160a78b4091bd8f9e7078a5357a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:04:15.690295 3431 tablet_service.cc:1467] Processing CreateTablet for tablet d46085a06902451c8a9ff416a4e5d713 (DEFAULT_TABLE table=test-table [id=696bc160a78b4091bd8f9e7078a5357a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:04:15.691497 3141 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d46085a06902451c8a9ff416a4e5d713. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:04:15.691497 3286 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d46085a06902451c8a9ff416a4e5d713. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:04:15.691501 3431 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d46085a06902451c8a9ff416a4e5d713. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:04:15.698632 3521 tablet_bootstrap.cc:492] T d46085a06902451c8a9ff416a4e5d713 P 28f3e5299c8d4d1db639b197e62cd43b: Bootstrap starting.
I20260811 19:04:15.699597 3522 tablet_bootstrap.cc:492] T d46085a06902451c8a9ff416a4e5d713 P 92e98b8bd63342249f98111ff0f94b87: Bootstrap starting.
I20260811 19:04:15.700999 3521 tablet_bootstrap.cc:654] T d46085a06902451c8a9ff416a4e5d713 P 28f3e5299c8d4d1db639b197e62cd43b: Neither blocks nor log segments found. Creating new log.
I20260811 19:04:15.701566 3522 tablet_bootstrap.cc:654] T d46085a06902451c8a9ff416a4e5d713 P 92e98b8bd63342249f98111ff0f94b87: Neither blocks nor log segments found. Creating new log.
I20260811 19:04:15.701817 3521 log.cc:824] T d46085a06902451c8a9ff416a4e5d713 P 28f3e5299c8d4d1db639b197e62cd43b: Log is configured to *not* fsync() on all Append() calls
I20260811 19:04:15.702273 3523 tablet_bootstrap.cc:492] T d46085a06902451c8a9ff416a4e5d713 P bc6ff4e413564c05bd52b2f966c12274: Bootstrap starting.
I20260811 19:04:15.702510 3522 log.cc:824] T d46085a06902451c8a9ff416a4e5d713 P 92e98b8bd63342249f98111ff0f94b87: Log is configured to *not* fsync() on all Append() calls
I20260811 19:04:15.704358 3522 tablet_bootstrap.cc:492] T d46085a06902451c8a9ff416a4e5d713 P 92e98b8bd63342249f98111ff0f94b87: No bootstrap required, opened a new log
I20260811 19:04:15.704496 3521 tablet_bootstrap.cc:492] T d46085a06902451c8a9ff416a4e5d713 P 28f3e5299c8d4d1db639b197e62cd43b: No bootstrap required, opened a new log
I20260811 19:04:15.704610 3522 ts_tablet_manager.cc:1397] T d46085a06902451c8a9ff416a4e5d713 P 92e98b8bd63342249f98111ff0f94b87: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260811 19:04:15.704659 3523 tablet_bootstrap.cc:654] T d46085a06902451c8a9ff416a4e5d713 P bc6ff4e413564c05bd52b2f966c12274: Neither blocks nor log segments found. Creating new log.
I20260811 19:04:15.704708 3521 ts_tablet_manager.cc:1397] T d46085a06902451c8a9ff416a4e5d713 P 28f3e5299c8d4d1db639b197e62cd43b: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 19:04:15.705545 3523 log.cc:824] T d46085a06902451c8a9ff416a4e5d713 P bc6ff4e413564c05bd52b2f966c12274: Log is configured to *not* fsync() on all Append() calls
I20260811 19:04:15.708356 3523 tablet_bootstrap.cc:492] T d46085a06902451c8a9ff416a4e5d713 P bc6ff4e413564c05bd52b2f966c12274: No bootstrap required, opened a new log
I20260811 19:04:15.708580 3523 ts_tablet_manager.cc:1397] T d46085a06902451c8a9ff416a4e5d713 P bc6ff4e413564c05bd52b2f966c12274: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 19:04:15.708375 3522 raft_consensus.cc:348] T d46085a06902451c8a9ff416a4e5d713 P 92e98b8bd63342249f98111ff0f94b87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92e98b8bd63342249f98111ff0f94b87" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 35509 } } peers { permanent_uuid: "28f3e5299c8d4d1db639b197e62cd43b" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 41667 } } peers { permanent_uuid: "bc6ff4e413564c05bd52b2f966c12274" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 43367 } }
I20260811 19:04:15.708698 3522 raft_consensus.cc:374] T d46085a06902451c8a9ff416a4e5d713 P 92e98b8bd63342249f98111ff0f94b87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:04:15.708757 3522 raft_consensus.cc:729] T d46085a06902451c8a9ff416a4e5d713 P 92e98b8bd63342249f98111ff0f94b87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92e98b8bd63342249f98111ff0f94b87, State: Initialized, Role: FOLLOWER
I20260811 19:04:15.709074 3521 raft_consensus.cc:348] T d46085a06902451c8a9ff416a4e5d713 P 28f3e5299c8d4d1db639b197e62cd43b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92e98b8bd63342249f98111ff0f94b87" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 35509 } } peers { permanent_uuid: "28f3e5299c8d4d1db639b197e62cd43b" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 41667 } } peers { permanent_uuid: "bc6ff4e413564c05bd52b2f966c12274" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 43367 } }
I20260811 19:04:15.709347 3521 raft_consensus.cc:374] T d46085a06902451c8a9ff416a4e5d713 P 28f3e5299c8d4d1db639b197e62cd43b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:04:15.709401 3521 raft_consensus.cc:729] T d46085a06902451c8a9ff416a4e5d713 P 28f3e5299c8d4d1db639b197e62cd43b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28f3e5299c8d4d1db639b197e62cd43b, State: Initialized, Role: FOLLOWER
I20260811 19:04:15.709324 3522 consensus_queue.cc:260] T d46085a06902451c8a9ff416a4e5d713 P 92e98b8bd63342249f98111ff0f94b87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92e98b8bd63342249f98111ff0f94b87" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 35509 } } peers { permanent_uuid: "28f3e5299c8d4d1db639b197e62cd43b" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 41667 } } peers { permanent_uuid: "bc6ff4e413564c05bd52b2f966c12274" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 43367 } }
I20260811 19:04:15.709872 3521 consensus_queue.cc:260] T d46085a06902451c8a9ff416a4e5d713 P 28f3e5299c8d4d1db639b197e62cd43b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92e98b8bd63342249f98111ff0f94b87" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 35509 } } peers { permanent_uuid: "28f3e5299c8d4d1db639b197e62cd43b" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 41667 } } peers { permanent_uuid: "bc6ff4e413564c05bd52b2f966c12274" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 43367 } }
I20260811 19:04:15.710603 3503 heartbeater.cc:503] Master 127.13.109.62:35469 was elected leader, sending a full tablet report...
I20260811 19:04:15.710935 3522 ts_tablet_manager.cc:1428] T d46085a06902451c8a9ff416a4e5d713 P 92e98b8bd63342249f98111ff0f94b87: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20260811 19:04:15.711329 3214 heartbeater.cc:503] Master 127.13.109.62:35469 was elected leader, sending a full tablet report...
I20260811 19:04:15.711987 3521 ts_tablet_manager.cc:1428] T d46085a06902451c8a9ff416a4e5d713 P 28f3e5299c8d4d1db639b197e62cd43b: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260811 19:04:15.712893 3523 raft_consensus.cc:348] T d46085a06902451c8a9ff416a4e5d713 P bc6ff4e413564c05bd52b2f966c12274 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92e98b8bd63342249f98111ff0f94b87" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 35509 } } peers { permanent_uuid: "28f3e5299c8d4d1db639b197e62cd43b" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 41667 } } peers { permanent_uuid: "bc6ff4e413564c05bd52b2f966c12274" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 43367 } }
I20260811 19:04:15.713212 3523 raft_consensus.cc:374] T d46085a06902451c8a9ff416a4e5d713 P bc6ff4e413564c05bd52b2f966c12274 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:04:15.713279 3523 raft_consensus.cc:729] T d46085a06902451c8a9ff416a4e5d713 P bc6ff4e413564c05bd52b2f966c12274 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc6ff4e413564c05bd52b2f966c12274, State: Initialized, Role: FOLLOWER
I20260811 19:04:15.713871 3523 consensus_queue.cc:260] T d46085a06902451c8a9ff416a4e5d713 P bc6ff4e413564c05bd52b2f966c12274 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92e98b8bd63342249f98111ff0f94b87" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 35509 } } peers { permanent_uuid: "28f3e5299c8d4d1db639b197e62cd43b" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 41667 } } peers { permanent_uuid: "bc6ff4e413564c05bd52b2f966c12274" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 43367 } }
I20260811 19:04:15.715088 3523 ts_tablet_manager.cc:1428] T d46085a06902451c8a9ff416a4e5d713 P bc6ff4e413564c05bd52b2f966c12274: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
I20260811 19:04:15.715692 3358 heartbeater.cc:503] Master 127.13.109.62:35469 was elected leader, sending a full tablet report...
W20260811 19:04:15.730667 3215 tablet.cc:2369] T d46085a06902451c8a9ff416a4e5d713 P 92e98b8bd63342249f98111ff0f94b87: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 19:04:15.755527 3528 raft_consensus.cc:482] T d46085a06902451c8a9ff416a4e5d713 P 28f3e5299c8d4d1db639b197e62cd43b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:04:15.755786 3528 raft_consensus.cc:504] T d46085a06902451c8a9ff416a4e5d713 P 28f3e5299c8d4d1db639b197e62cd43b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92e98b8bd63342249f98111ff0f94b87" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 35509 } } peers { permanent_uuid: "28f3e5299c8d4d1db639b197e62cd43b" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 41667 } } peers { permanent_uuid: "bc6ff4e413564c05bd52b2f966c12274" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 43367 } }
I20260811 19:04:15.757021 3528 leader_election.cc:288] T d46085a06902451c8a9ff416a4e5d713 P 28f3e5299c8d4d1db639b197e62cd43b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 92e98b8bd63342249f98111ff0f94b87 (127.13.109.1:35509), bc6ff4e413564c05bd52b2f966c12274 (127.13.109.2:43367)
I20260811 19:04:15.761610 3505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:15.757375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.1:35509 (local address 127.13.109.3:39383)
0811 19:04:15.757513 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:15.757533 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 19:04:15.757651 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:04:15.758116 (+ 465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:04:15.758120 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:04:15.758141 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:04:15.758290 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:15.758301 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:15.759128 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:15.759132 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:15.760241 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:15.760251 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:15.761144 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:15.761147 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:15.761271 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:15.761298 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 19:04:15.761381 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 19:04:15.761460 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260811 19:04:15.761585 3516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:15.757463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.3:39383 (local address 127.13.109.1:35509)
0811 19:04:15.757602 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:15.757607 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:15.757623 (+ 16us) server_negotiation.cc:407] Connection header received
0811 19:04:15.757838 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:15.757842 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:15.757903 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:15.757976 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:04:15.758408 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:15.759016 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:15.760385 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:15.761022 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:15.761115 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:15.761158 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:15.761410 (+ 252us) server_negotiation.cc:299] Negotiation successful
0811 19:04:15.761475 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 19:04:15.762319 3530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:15.757671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.2:43367 (local address 127.13.109.3:54925)
0811 19:04:15.758180 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:15.758195 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 19:04:15.758330 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:04:15.758687 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:04:15.758690 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:04:15.758707 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:04:15.758834 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:15.758839 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:15.759851 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:15.759853 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:15.760838 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:15.760850 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:15.761899 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:15.761925 (+ 26us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:15.762027 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:15.762050 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 19:04:15.762137 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 19:04:15.762205 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":190,"threads_started":1}
I20260811 19:04:15.762389 3520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:15.757878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.109.3:54925 (local address 127.13.109.2:43367)
0811 19:04:15.758013 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:15.758018 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:15.758293 (+ 275us) server_negotiation.cc:407] Connection header received
0811 19:04:15.758503 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:15.758506 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:15.758571 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:15.758674 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:04:15.758957 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:15.759708 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:15.760972 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:15.761538 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:15.761678 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:15.761742 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:15.762180 (+ 438us) server_negotiation.cc:299] Negotiation successful
0811 19:04:15.762275 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 19:04:15.762477 3164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d46085a06902451c8a9ff416a4e5d713" candidate_uuid: "28f3e5299c8d4d1db639b197e62cd43b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92e98b8bd63342249f98111ff0f94b87" is_pre_election: true
I20260811 19:04:15.762882 3164 raft_consensus.cc:2454] T d46085a06902451c8a9ff416a4e5d713 P 92e98b8bd63342249f98111ff0f94b87 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28f3e5299c8d4d1db639b197e62cd43b in term 0.
I20260811 19:04:15.762976 3308 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d46085a06902451c8a9ff416a4e5d713" candidate_uuid: "28f3e5299c8d4d1db639b197e62cd43b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc6ff4e413564c05bd52b2f966c12274" is_pre_election: true
I20260811 19:04:15.763309 3308 raft_consensus.cc:2454] T d46085a06902451c8a9ff416a4e5d713 P bc6ff4e413564c05bd52b2f966c12274 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28f3e5299c8d4d1db639b197e62cd43b in term 0.
I20260811 19:04:15.763460 3381 leader_election.cc:302] T d46085a06902451c8a9ff416a4e5d713 P 28f3e5299c8d4d1db639b197e62cd43b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28f3e5299c8d4d1db639b197e62cd43b, 92e98b8bd63342249f98111ff0f94b87; no voters:
I20260811 19:04:15.763754 3528 raft_consensus.cc:2781] T d46085a06902451c8a9ff416a4e5d713 P 28f3e5299c8d4d1db639b197e62cd43b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 19:04:15.763849 3528 raft_consensus.cc:482] T d46085a06902451c8a9ff416a4e5d713 P 28f3e5299c8d4d1db639b197e62cd43b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 19:04:15.763895 3528 raft_consensus.cc:3037] T d46085a06902451c8a9ff416a4e5d713 P 28f3e5299c8d4d1db639b197e62cd43b [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:04:15.764981 3528 raft_consensus.cc:504] T d46085a06902451c8a9ff416a4e5d713 P 28f3e5299c8d4d1db639b197e62cd43b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92e98b8bd63342249f98111ff0f94b87" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 35509 } } peers { permanent_uuid: "28f3e5299c8d4d1db639b197e62cd43b" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 41667 } } peers { permanent_uuid: "bc6ff4e413564c05bd52b2f966c12274" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 43367 } }
I20260811 19:04:15.765362 3528 leader_election.cc:288] T d46085a06902451c8a9ff416a4e5d713 P 28f3e5299c8d4d1db639b197e62cd43b [CANDIDATE]: Term 1 election: Requested vote from peers 92e98b8bd63342249f98111ff0f94b87 (127.13.109.1:35509), bc6ff4e413564c05bd52b2f966c12274 (127.13.109.2:43367)
I20260811 19:04:15.765770 3308 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d46085a06902451c8a9ff416a4e5d713" candidate_uuid: "28f3e5299c8d4d1db639b197e62cd43b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc6ff4e413564c05bd52b2f966c12274"
I20260811 19:04:15.765758 3164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d46085a06902451c8a9ff416a4e5d713" candidate_uuid: "28f3e5299c8d4d1db639b197e62cd43b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92e98b8bd63342249f98111ff0f94b87"
I20260811 19:04:15.765972 3308 raft_consensus.cc:3037] T d46085a06902451c8a9ff416a4e5d713 P bc6ff4e413564c05bd52b2f966c12274 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:04:15.766109 3164 raft_consensus.cc:3037] T d46085a06902451c8a9ff416a4e5d713 P 92e98b8bd63342249f98111ff0f94b87 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:04:15.767128 3308 raft_consensus.cc:2454] T d46085a06902451c8a9ff416a4e5d713 P bc6ff4e413564c05bd52b2f966c12274 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28f3e5299c8d4d1db639b197e62cd43b in term 1.
I20260811 19:04:15.767199 3164 raft_consensus.cc:2454] T d46085a06902451c8a9ff416a4e5d713 P 92e98b8bd63342249f98111ff0f94b87 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28f3e5299c8d4d1db639b197e62cd43b in term 1.
I20260811 19:04:15.767488 3381 leader_election.cc:302] T d46085a06902451c8a9ff416a4e5d713 P 28f3e5299c8d4d1db639b197e62cd43b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28f3e5299c8d4d1db639b197e62cd43b, bc6ff4e413564c05bd52b2f966c12274; no voters:
I20260811 19:04:15.767689 3528 raft_consensus.cc:2781] T d46085a06902451c8a9ff416a4e5d713 P 28f3e5299c8d4d1db639b197e62cd43b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:04:15.767936 3528 raft_consensus.cc:686] T d46085a06902451c8a9ff416a4e5d713 P 28f3e5299c8d4d1db639b197e62cd43b [term 1 LEADER]: Becoming Leader. State: Replica: 28f3e5299c8d4d1db639b197e62cd43b, State: Running, Role: LEADER
I20260811 19:04:15.768322 3528 consensus_queue.cc:237] T d46085a06902451c8a9ff416a4e5d713 P 28f3e5299c8d4d1db639b197e62cd43b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92e98b8bd63342249f98111ff0f94b87" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 35509 } } peers { permanent_uuid: "28f3e5299c8d4d1db639b197e62cd43b" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 41667 } } peers { permanent_uuid: "bc6ff4e413564c05bd52b2f966c12274" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 43367 } }
I20260811 19:04:15.771723 3006 catalog_manager.cc:5495] T d46085a06902451c8a9ff416a4e5d713 P 28f3e5299c8d4d1db639b197e62cd43b reported cstate change: term changed from 0 to 1, leader changed from <none> to 28f3e5299c8d4d1db639b197e62cd43b (127.13.109.3). New cstate: current_term: 1 leader_uuid: "28f3e5299c8d4d1db639b197e62cd43b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92e98b8bd63342249f98111ff0f94b87" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 35509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28f3e5299c8d4d1db639b197e62cd43b" member_type: VOTER last_known_addr { host: "127.13.109.3" port: 41667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc6ff4e413564c05bd52b2f966c12274" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 43367 } health_report { overall_health: UNKNOWN } } }
I20260811 19:04:15.786106 3518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:15.782624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43772 (local address 127.13.109.3:41667)
0811 19:04:15.782766 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:15.782769 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:15.782784 (+ 15us) server_negotiation.cc:407] Connection header received
0811 19:04:15.782864 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:15.782866 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:15.782926 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:15.783019 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:04:15.783243 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:15.783740 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:15.784542 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:15.784766 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:15.784844 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:15.785177 (+ 333us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:04:15.785637 (+ 460us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:04:15.785728 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:15.785861 (+ 133us) server_negotiation.cc:299] Negotiation successful
0811 19:04:15.785925 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 19:04:15.792550 3308 raft_consensus.cc:1261] T d46085a06902451c8a9ff416a4e5d713 P bc6ff4e413564c05bd52b2f966c12274 [term 1 FOLLOWER]: Refusing update from remote peer 28f3e5299c8d4d1db639b197e62cd43b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:04:15.792608 3164 raft_consensus.cc:1261] T d46085a06902451c8a9ff416a4e5d713 P 92e98b8bd63342249f98111ff0f94b87 [term 1 FOLLOWER]: Refusing update from remote peer 28f3e5299c8d4d1db639b197e62cd43b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:04:15.793452 3532 consensus_queue.cc:1035] T d46085a06902451c8a9ff416a4e5d713 P 28f3e5299c8d4d1db639b197e62cd43b [LEADER]: Connected to new peer: Peer: permanent_uuid: "92e98b8bd63342249f98111ff0f94b87" member_type: VOTER last_known_addr { host: "127.13.109.1" port: 35509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:04:15.793697 3528 consensus_queue.cc:1035] T d46085a06902451c8a9ff416a4e5d713 P 28f3e5299c8d4d1db639b197e62cd43b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc6ff4e413564c05bd52b2f966c12274" member_type: VOTER last_known_addr { host: "127.13.109.2" port: 43367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:04:15.801748 3535 mvcc.cc:205] Tried to move back new op lower bound from 7317401828517691392 to 7317401828430118912. Current Snapshot: MvccSnapshot[applied={T|T < 7317401828517691392}]
I20260811 19:04:15.802781 3536 mvcc.cc:205] Tried to move back new op lower bound from 7317401828517691392 to 7317401828430118912. Current Snapshot: MvccSnapshot[applied={T|T < 7317401828517691392}]
I20260811 19:04:15.812263 3006 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38614:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:04:15.812487 3006 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38614:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:04:15.815541 3006 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6d39ae7810d8493a8cbb04aaca8ec900: Sending DeleteTablet for 3 replicas of tablet d46085a06902451c8a9ff416a4e5d713
I20260811 19:04:15.816550 3431 tablet_service.cc:1514] Processing DeleteTablet for tablet d46085a06902451c8a9ff416a4e5d713 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:04:15 UTC) from {username='slave'} at 127.0.0.1:43762
I20260811 19:04:15.816550 3286 tablet_service.cc:1514] Processing DeleteTablet for tablet d46085a06902451c8a9ff416a4e5d713 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:04:15 UTC) from {username='slave'} at 127.0.0.1:58312
I20260811 19:04:15.817204 3544 tablet_replica.cc:330] T d46085a06902451c8a9ff416a4e5d713 P bc6ff4e413564c05bd52b2f966c12274: stopping tablet replica
I20260811 19:04:15.817188 3141 tablet_service.cc:1514] Processing DeleteTablet for tablet d46085a06902451c8a9ff416a4e5d713 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:04:15 UTC) from {username='slave'} at 127.0.0.1:52312
I20260811 19:04:15.817538 3546 tablet_replica.cc:330] T d46085a06902451c8a9ff416a4e5d713 P 28f3e5299c8d4d1db639b197e62cd43b: stopping tablet replica
I20260811 19:04:15.817678 3547 tablet_replica.cc:330] T d46085a06902451c8a9ff416a4e5d713 P 92e98b8bd63342249f98111ff0f94b87: stopping tablet replica
I20260811 19:04:15.817978 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 3074
I20260811 19:04:15.818928 3544 raft_consensus.cc:2229] T d46085a06902451c8a9ff416a4e5d713 P bc6ff4e413564c05bd52b2f966c12274 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:04:15.819217 3546 raft_consensus.cc:2229] T d46085a06902451c8a9ff416a4e5d713 P 28f3e5299c8d4d1db639b197e62cd43b [term 1 LEADER]: Raft consensus shutting down.
I20260811 19:04:15.819681 3546 raft_consensus.cc:2258] T d46085a06902451c8a9ff416a4e5d713 P 28f3e5299c8d4d1db639b197e62cd43b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:04:15.820088 3544 raft_consensus.cc:2258] T d46085a06902451c8a9ff416a4e5d713 P bc6ff4e413564c05bd52b2f966c12274 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:04:15.823010 3544 ts_tablet_manager.cc:1905] T d46085a06902451c8a9ff416a4e5d713 P bc6ff4e413564c05bd52b2f966c12274: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:04:15.827064 3544 ts_tablet_manager.cc:1918] T d46085a06902451c8a9ff416a4e5d713 P bc6ff4e413564c05bd52b2f966c12274: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:04:15.827306 3544 log.cc:1196] T d46085a06902451c8a9ff416a4e5d713 P bc6ff4e413564c05bd52b2f966c12274: Deleting WAL directory at /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474792496492-13748-0/minicluster-data/ts-1/wal/wals/d46085a06902451c8a9ff416a4e5d713
I20260811 19:04:15.827786 3546 ts_tablet_manager.cc:1905] T d46085a06902451c8a9ff416a4e5d713 P 28f3e5299c8d4d1db639b197e62cd43b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:04:15.827836 3544 ts_tablet_manager.cc:1939] T d46085a06902451c8a9ff416a4e5d713 P bc6ff4e413564c05bd52b2f966c12274: Deleting consensus metadata
I20260811 19:04:15.829589 2992 catalog_manager.cc:4841] TS bc6ff4e413564c05bd52b2f966c12274 (127.13.109.2:43367): tablet d46085a06902451c8a9ff416a4e5d713 (table test-table [id=696bc160a78b4091bd8f9e7078a5357a]) successfully deleted
I20260811 19:04:15.831691 3546 ts_tablet_manager.cc:1918] T d46085a06902451c8a9ff416a4e5d713 P 28f3e5299c8d4d1db639b197e62cd43b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:04:15.831837 3546 log.cc:1196] T d46085a06902451c8a9ff416a4e5d713 P 28f3e5299c8d4d1db639b197e62cd43b: Deleting WAL directory at /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474792496492-13748-0/minicluster-data/ts-2/wal/wals/d46085a06902451c8a9ff416a4e5d713
I20260811 19:04:15.832198 3546 ts_tablet_manager.cc:1939] T d46085a06902451c8a9ff416a4e5d713 P 28f3e5299c8d4d1db639b197e62cd43b: Deleting consensus metadata
I20260811 19:04:15.833761 2988 catalog_manager.cc:4841] TS 28f3e5299c8d4d1db639b197e62cd43b (127.13.109.3:41667): tablet d46085a06902451c8a9ff416a4e5d713 (table test-table [id=696bc160a78b4091bd8f9e7078a5357a]) successfully deleted
W20260811 19:04:15.834511 2992 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.109.1:35509 (error 108)
I20260811 19:04:15.835469 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 3217
I20260811 19:04:15.835764 3519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:15.835420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.1:35509 (local address 127.0.0.1:52320)
0811 19:04:15.835572 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:15.835656 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.1:35509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260811 19:04:15.838286 2992 catalog_manager.cc:4584] TS 92e98b8bd63342249f98111ff0f94b87 (127.13.109.1:35509): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d46085a06902451c8a9ff416a4e5d713: Network error: Client connection negotiation failed: client connection to 127.13.109.1:35509: connect: Connection refused (error 111)
I20260811 19:04:15.845461 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 3362
I20260811 19:04:15.855463 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 2973
2026-08-11T19:04:15Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1133 ms)
[----------] 6 tests from EncryptionPolicyTest (11484 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474792496492-13748-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:04:15 dist-test-slave-0kls krb5kdc[3552](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:04:15 dist-test-slave-0kls krb5kdc[3552](info): set up 2 sockets
Aug 11 19:04:15 dist-test-slave-0kls krb5kdc[3552](info): commencing operation
krb5kdc: starting...
W20260811 19:04:16.966457 13748 mini_kdc.cc:121] Time spent starting KDC: real 1.085s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474792496492-13748-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474792496492-13748-0/krb5kdc/test-user.keytab.
Aug 11 19:04:17 dist-test-slave-0kls krb5kdc[3552](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475057, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:04:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:04:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474792496492-13748-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474792496492-13748-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 19:04:17.108088 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474792496492-13748-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474792496492-13748-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474792496492-13748-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44489
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474792496492-13748-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44489
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:04:17.228555 3566 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:04:17.228876 3566 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:04:17.228920 3566 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:04:17.233031 3566 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:04:17.233114 3566 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:04:17.233135 3566 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:04:17.233153 3566 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:04:17.233170 3566 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:04:17.238976 3566 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474792496492-13748-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474792496492-13748-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44489
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474792496492-13748-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44489
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3566
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474792496492-13748-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:04:17.240188 3566 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:04:17.241276 3566 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:04:17.250245 3572 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:04:17.250293 3571 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:04:17.250258 3574 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:04:17.250550 3566 server_base.cc:1034] running on GCE node
I20260811 19:04:17.251252 3566 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:04:17.252480 3566 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:04:17.253693 3566 hybrid_clock.cc:648] HybridClock initialized: now 1786475057253681 us; error 32 us; skew 500 ppm
Aug 11 19:04:17 dist-test-slave-0kls krb5kdc[3552](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475057, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:04:17.257066 3566 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 19:04:17.258158 3566 webserver.cc:460] Webserver started at http://127.0.0.1:43031/ using document root <none> and password file <none>
I20260811 19:04:17.258802 3566 fs_manager.cc:359] Metadata directory not provided
I20260811 19:04:17.258904 3566 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:04:17.259171 3566 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:04:17.261478 3566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474792496492-13748-0/minicluster-data/master-0/data/instance:
uuid: "ec033a89eebe4fa9a026e08daca1ed1a"
format_stamp: "Formatted at 2026-08-11 19:04:17 on dist-test-slave-0kls"
server_key: "475ff9c8ee726256d80d01e74c1317fe"
server_key_iv: "4b5caad0a06a60bb4dfeb4e12ac1278e"
server_key_version: "encryptionkey@0"
I20260811 19:04:17.262166 3566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474792496492-13748-0/minicluster-data/master-0/wal/instance:
uuid: "ec033a89eebe4fa9a026e08daca1ed1a"
format_stamp: "Formatted at 2026-08-11 19:04:17 on dist-test-slave-0kls"
server_key: "475ff9c8ee726256d80d01e74c1317fe"
server_key_iv: "4b5caad0a06a60bb4dfeb4e12ac1278e"
server_key_version: "encryptionkey@0"
I20260811 19:04:17.266300 3566 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 19:04:17.268930 3582 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:04:17.270491 3566 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 19:04:17.270651 3566 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474792496492-13748-0/minicluster-data/master-0/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474792496492-13748-0/minicluster-data/master-0/wal
uuid: "ec033a89eebe4fa9a026e08daca1ed1a"
format_stamp: "Formatted at 2026-08-11 19:04:17 on dist-test-slave-0kls"
server_key: "475ff9c8ee726256d80d01e74c1317fe"
server_key_iv: "4b5caad0a06a60bb4dfeb4e12ac1278e"
server_key_version: "encryptionkey@0"
I20260811 19:04:17.270781 3566 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474792496492-13748-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474792496492-13748-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474792496492-13748-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:04:17.282898 3566 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:04:17.286262 3566 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:04:17.286492 3566 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:04:17.294224 3566 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44489
I20260811 19:04:17.294312 3643 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44489 every 8 connection(s)
I20260811 19:04:17.295328 3566 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
I20260811 19:04:17.296391 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 3566
I20260811 19:04:17.296543 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474792496492-13748-0/minicluster-data/master-0/wal/instance
I20260811 19:04:17.296880 13748 external_mini_cluster.cc:1428] Setting key 6d75d3e2c458487cf2272bcd66393dd4
I20260811 19:04:17.298936 3644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:04:17.306495 3644 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec033a89eebe4fa9a026e08daca1ed1a: Bootstrap starting.
Aug 11 19:04:17 dist-test-slave-0kls krb5kdc[3552](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475057, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 19:04:17.309499 3644 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ec033a89eebe4fa9a026e08daca1ed1a: Neither blocks nor log segments found. Creating new log.
I20260811 19:04:17.310554 3644 log.cc:824] T 00000000000000000000000000000000 P ec033a89eebe4fa9a026e08daca1ed1a: Log is configured to *not* fsync() on all Append() calls
I20260811 19:04:17.312985 3644 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec033a89eebe4fa9a026e08daca1ed1a: No bootstrap required, opened a new log
I20260811 19:04:17.313423 3647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:17.298426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34839 (local address 127.0.0.1:44489)
0811 19:04:17.298876 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:17.298899 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:17.298938 (+ 39us) server_negotiation.cc:407] Connection header received
0811 19:04:17.299680 (+ 742us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:17.299697 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:17.300055 (+ 358us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:17.300361 (+ 306us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:17.301327 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:17.302445 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:17.303643 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:17.304132 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:17.304255 (+ 123us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:17.307570 (+ 3315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:04:17.307576 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:04:17.307591 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:04:17.307638 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:04:17.309663 (+ 2025us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:17.310370 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:17.310376 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:17.310382 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:17.310535 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:17.310985 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:17.310988 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:17.310989 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:17.311619 (+ 630us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:04:17.311803 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:17.312269 (+ 466us) server_negotiation.cc:299] Negotiation successful
0811 19:04:17.312556 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":162,"threads_started":1}
I20260811 19:04:17.317045 13748 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 19:04:17.317555 3644 raft_consensus.cc:348] T 00000000000000000000000000000000 P ec033a89eebe4fa9a026e08daca1ed1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec033a89eebe4fa9a026e08daca1ed1a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44489 } }
I20260811 19:04:17.317899 3644 raft_consensus.cc:374] T 00000000000000000000000000000000 P ec033a89eebe4fa9a026e08daca1ed1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:04:17.317978 3644 raft_consensus.cc:729] T 00000000000000000000000000000000 P ec033a89eebe4fa9a026e08daca1ed1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec033a89eebe4fa9a026e08daca1ed1a, State: Initialized, Role: FOLLOWER
I20260811 19:04:17.318500 3644 consensus_queue.cc:260] T 00000000000000000000000000000000 P ec033a89eebe4fa9a026e08daca1ed1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec033a89eebe4fa9a026e08daca1ed1a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44489 } }
I20260811 19:04:17.318698 3644 raft_consensus.cc:388] T 00000000000000000000000000000000 P ec033a89eebe4fa9a026e08daca1ed1a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:04:17.318782 3644 raft_consensus.cc:482] T 00000000000000000000000000000000 P ec033a89eebe4fa9a026e08daca1ed1a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:04:17.318892 3644 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ec033a89eebe4fa9a026e08daca1ed1a [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:04:17.320253 3644 raft_consensus.cc:504] T 00000000000000000000000000000000 P ec033a89eebe4fa9a026e08daca1ed1a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec033a89eebe4fa9a026e08daca1ed1a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44489 } }
I20260811 19:04:17.320681 3644 leader_election.cc:302] T 00000000000000000000000000000000 P ec033a89eebe4fa9a026e08daca1ed1a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ec033a89eebe4fa9a026e08daca1ed1a; no voters:
I20260811 19:04:17.321062 3644 leader_election.cc:288] T 00000000000000000000000000000000 P ec033a89eebe4fa9a026e08daca1ed1a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:04:17.321173 3655 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ec033a89eebe4fa9a026e08daca1ed1a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:04:17.321404 3655 raft_consensus.cc:686] T 00000000000000000000000000000000 P ec033a89eebe4fa9a026e08daca1ed1a [term 1 LEADER]: Becoming Leader. State: Replica: ec033a89eebe4fa9a026e08daca1ed1a, State: Running, Role: LEADER
I20260811 19:04:17.321724 3655 consensus_queue.cc:237] T 00000000000000000000000000000000 P ec033a89eebe4fa9a026e08daca1ed1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec033a89eebe4fa9a026e08daca1ed1a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44489 } }
I20260811 19:04:17.322549 3644 sys_catalog.cc:564] T 00000000000000000000000000000000 P ec033a89eebe4fa9a026e08daca1ed1a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:04:17.323544 3656 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec033a89eebe4fa9a026e08daca1ed1a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ec033a89eebe4fa9a026e08daca1ed1a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec033a89eebe4fa9a026e08daca1ed1a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44489 } } }
I20260811 19:04:17.323680 3656 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec033a89eebe4fa9a026e08daca1ed1a [sys.catalog]: This master's current role is: LEADER
I20260811 19:04:17.324002 3657 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec033a89eebe4fa9a026e08daca1ed1a [sys.catalog]: SysCatalogTable state changed. Reason: New leader ec033a89eebe4fa9a026e08daca1ed1a. Latest consensus state: current_term: 1 leader_uuid: "ec033a89eebe4fa9a026e08daca1ed1a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec033a89eebe4fa9a026e08daca1ed1a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44489 } } }
I20260811 19:04:17.324136 3657 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec033a89eebe4fa9a026e08daca1ed1a [sys.catalog]: This master's current role is: LEADER
I20260811 19:04:17.324416 3664 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:04:17.328689 3664 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:04:17.329875 3647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:17.319610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34409 (local address 127.0.0.1:44489)
0811 19:04:17.319767 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:17.319771 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:17.319941 (+ 170us) server_negotiation.cc:407] Connection header received
0811 19:04:17.320069 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:17.320071 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:17.320115 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:17.320181 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:17.321069 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:17.321890 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:17.323040 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:17.323454 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:17.323572 (+ 118us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:17.325544 (+ 1972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:04:17.325548 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:04:17.325551 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:04:17.325579 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:04:17.327223 (+ 1644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:17.327977 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:17.327981 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:17.327983 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:17.328092 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:17.328486 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:17.328490 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:17.328491 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:17.328734 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:04:17.328881 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:17.329587 (+ 706us) server_negotiation.cc:299] Negotiation successful
0811 19:04:17.329690 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 19:04:17.336076 3664 catalog_manager.cc:1347] Generated new cluster ID: 4d470e854cf54790ad28d98c2abfb168
I20260811 19:04:17.336165 3664 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:04:17.348732 3664 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:04:17.350085 3664 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:04:17.355525 3664 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ec033a89eebe4fa9a026e08daca1ed1a: Generated new TSK 0
I20260811 19:04:17.356316 3664 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 19:04:17.366925 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 3566
W20260811 19:04:17.376188 3549 connection.cc:535] client connection to 127.0.0.1:44489 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44489): Connection reset by peer (error 104)
2026-08-11T19:04:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1509 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474792496492-13748-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:04:17 dist-test-slave-0kls krb5kdc[3678](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:04:17 dist-test-slave-0kls krb5kdc[3678](info): set up 2 sockets
Aug 11 19:04:17 dist-test-slave-0kls krb5kdc[3678](info): commencing operation
krb5kdc: starting...
W20260811 19:04:18.467657 13748 mini_kdc.cc:121] Time spent starting KDC: real 1.077s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474792496492-13748-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474792496492-13748-0/krb5kdc/test-user.keytab.
Aug 11 19:04:18 dist-test-slave-0kls krb5kdc[3678](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475058, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:04:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:04:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474792496492-13748-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474792496492-13748-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 19:04:18.604866 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474792496492-13748-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474792496492-13748-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474792496492-13748-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44127
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474792496492-13748-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44127
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:04:18.730610 3695 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:04:18.730995 3695 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:04:18.731096 3695 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:04:18.735462 3695 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:04:18.735565 3695 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:04:18.735602 3695 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:04:18.735657 3695 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:04:18.735692 3695 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:04:18.741429 3695 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474792496492-13748-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474792496492-13748-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44127
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474792496492-13748-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44127
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3695
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474792496492-13748-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:04:18.742805 3695 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:04:18.743938 3695 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:04:18.753715 3706 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:04:18.754029 3695 server_base.cc:1034] running on GCE node
W20260811 19:04:18.753741 3703 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:04:18.753729 3704 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:04:18.754810 3695 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:04:18.756078 3695 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:04:18.757288 3695 hybrid_clock.cc:648] HybridClock initialized: now 1786475058757274 us; error 28 us; skew 500 ppm
Aug 11 19:04:18 dist-test-slave-0kls krb5kdc[3678](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475058, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:04:18.760260 3695 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 19:04:18.761243 3695 webserver.cc:460] Webserver started at http://127.0.0.1:36423/ using document root <none> and password file <none>
I20260811 19:04:18.761916 3695 fs_manager.cc:359] Metadata directory not provided
I20260811 19:04:18.762017 3695 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:04:18.762281 3695 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:04:18.764489 3695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474792496492-13748-0/minicluster-data/master-0/data/instance:
uuid: "7e6f4d9caf354653bee74421cd1106dc"
format_stamp: "Formatted at 2026-08-11 19:04:18 on dist-test-slave-0kls"
server_key: "edd81d0f994438d87ce93af50249e7a5"
server_key_iv: "892a20070c96548e4ae4df92eea04e95"
server_key_version: "encryptionkey@0"
I20260811 19:04:18.765115 3695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474792496492-13748-0/minicluster-data/master-0/wal/instance:
uuid: "7e6f4d9caf354653bee74421cd1106dc"
format_stamp: "Formatted at 2026-08-11 19:04:18 on dist-test-slave-0kls"
server_key: "edd81d0f994438d87ce93af50249e7a5"
server_key_iv: "892a20070c96548e4ae4df92eea04e95"
server_key_version: "encryptionkey@0"
I20260811 19:04:18.769294 3695 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 19:04:18.772130 3713 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:04:18.773969 3695 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 19:04:18.774127 3695 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474792496492-13748-0/minicluster-data/master-0/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474792496492-13748-0/minicluster-data/master-0/wal
uuid: "7e6f4d9caf354653bee74421cd1106dc"
format_stamp: "Formatted at 2026-08-11 19:04:18 on dist-test-slave-0kls"
server_key: "edd81d0f994438d87ce93af50249e7a5"
server_key_iv: "892a20070c96548e4ae4df92eea04e95"
server_key_version: "encryptionkey@0"
I20260811 19:04:18.774259 3695 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474792496492-13748-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474792496492-13748-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474792496492-13748-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:04:18.790902 3695 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:04:18.794426 3695 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:04:18.794597 3695 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:04:18.801714 3695 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44127
I20260811 19:04:18.801808 3772 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44127 every 8 connection(s)
I20260811 19:04:18.802817 3695 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
I20260811 19:04:18.803309 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 3695
I20260811 19:04:18.803463 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474792496492-13748-0/minicluster-data/master-0/wal/instance
I20260811 19:04:18.803772 13748 external_mini_cluster.cc:1428] Setting key c7f23725b36e12f256c310df2863cd8f
I20260811 19:04:18.806211 3773 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:04:18.812027 3773 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e6f4d9caf354653bee74421cd1106dc: Bootstrap starting.
I20260811 19:04:18.813088 3776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:18.805240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39405 (local address 127.0.0.1:44127)
0811 19:04:18.805703 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:18.805719 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:18.805756 (+ 37us) server_negotiation.cc:407] Connection header received
0811 19:04:18.806550 (+ 794us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:18.806569 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:18.806913 (+ 344us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:18.807283 (+ 370us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:18.807733 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:18.808765 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:18.809707 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:18.810269 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:18.810449 (+ 180us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:18.811420 (+ 971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:04:18.811425 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:04:18.811437 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:04:18.811839 (+ 402us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:04:18.811894 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:04:18.811917 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:04:18.812071 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:18.812228 (+ 157us) server_negotiation.cc:299] Negotiation successful
0811 19:04:18.812377 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":138,"threads_started":1}
I20260811 19:04:18.814723 3773 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7e6f4d9caf354653bee74421cd1106dc: Neither blocks nor log segments found. Creating new log.
I20260811 19:04:18.815635 3773 log.cc:824] T 00000000000000000000000000000000 P 7e6f4d9caf354653bee74421cd1106dc: Log is configured to *not* fsync() on all Append() calls
I20260811 19:04:18.816360 13748 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 19:04:18.817770 3773 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e6f4d9caf354653bee74421cd1106dc: No bootstrap required, opened a new log
I20260811 19:04:18.822897 3776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:18.818660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57603 (local address 127.0.0.1:44127)
0811 19:04:18.818796 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:18.818801 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:18.818945 (+ 144us) server_negotiation.cc:407] Connection header received
0811 19:04:18.819027 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:18.819029 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:18.819076 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:18.819170 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:18.819422 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:18.820273 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:18.821079 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:18.821419 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:18.821528 (+ 109us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:18.822422 (+ 894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:04:18.822425 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:04:18.822428 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:04:18.822444 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:04:18.822471 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:04:18.822484 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:04:18.822586 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:18.822730 (+ 144us) server_negotiation.cc:299] Negotiation successful
0811 19:04:18.822773 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 19:04:18.822662 3773 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7e6f4d9caf354653bee74421cd1106dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e6f4d9caf354653bee74421cd1106dc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44127 } }
I20260811 19:04:18.823004 3773 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7e6f4d9caf354653bee74421cd1106dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:04:18.823063 3773 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7e6f4d9caf354653bee74421cd1106dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e6f4d9caf354653bee74421cd1106dc, State: Initialized, Role: FOLLOWER
I20260811 19:04:18.823552 3773 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7e6f4d9caf354653bee74421cd1106dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e6f4d9caf354653bee74421cd1106dc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44127 } }
I20260811 19:04:18.823696 3773 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7e6f4d9caf354653bee74421cd1106dc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:04:18.823791 3773 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7e6f4d9caf354653bee74421cd1106dc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:04:18.823904 3773 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7e6f4d9caf354653bee74421cd1106dc [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:04:18.824934 3773 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7e6f4d9caf354653bee74421cd1106dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e6f4d9caf354653bee74421cd1106dc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44127 } }
I20260811 19:04:18.825313 3773 leader_election.cc:302] T 00000000000000000000000000000000 P 7e6f4d9caf354653bee74421cd1106dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7e6f4d9caf354653bee74421cd1106dc; no voters:
I20260811 19:04:18.825667 3773 leader_election.cc:288] T 00000000000000000000000000000000 P 7e6f4d9caf354653bee74421cd1106dc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:04:18.825785 3786 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7e6f4d9caf354653bee74421cd1106dc [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:04:18.826085 3786 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7e6f4d9caf354653bee74421cd1106dc [term 1 LEADER]: Becoming Leader. State: Replica: 7e6f4d9caf354653bee74421cd1106dc, State: Running, Role: LEADER
I20260811 19:04:18.826406 3786 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7e6f4d9caf354653bee74421cd1106dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e6f4d9caf354653bee74421cd1106dc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44127 } }
I20260811 19:04:18.827252 3773 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7e6f4d9caf354653bee74421cd1106dc [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:04:18.828287 3788 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e6f4d9caf354653bee74421cd1106dc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7e6f4d9caf354653bee74421cd1106dc. Latest consensus state: current_term: 1 leader_uuid: "7e6f4d9caf354653bee74421cd1106dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e6f4d9caf354653bee74421cd1106dc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44127 } } }
I20260811 19:04:18.828431 3788 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e6f4d9caf354653bee74421cd1106dc [sys.catalog]: This master's current role is: LEADER
I20260811 19:04:18.828747 3787 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e6f4d9caf354653bee74421cd1106dc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7e6f4d9caf354653bee74421cd1106dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e6f4d9caf354653bee74421cd1106dc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44127 } } }
I20260811 19:04:18.828831 3787 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e6f4d9caf354653bee74421cd1106dc [sys.catalog]: This master's current role is: LEADER
I20260811 19:04:18.828819 3795 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:04:18.832226 3795 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:04:18.838290 3795 catalog_manager.cc:1347] Generated new cluster ID: 4dd3ca12e0ce46f280b53e61db448597
I20260811 19:04:18.838369 3795 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:04:18.848979 3795 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:04:18.850452 3795 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:04:18.861451 3795 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7e6f4d9caf354653bee74421cd1106dc: Generated new TSK 0
I20260811 19:04:18.862473 3795 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 19:04:18.868040 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 3695
W20260811 19:04:18.877942 3676 connection.cc:535] client connection to 127.0.0.1:44127 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44127): Connection reset by peer (error 104)
2026-08-11T19:04:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1502 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474792496492-13748-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:04:18 dist-test-slave-0kls krb5kdc[3808](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:04:18 dist-test-slave-0kls krb5kdc[3808](info): set up 2 sockets
Aug 11 19:04:18 dist-test-slave-0kls krb5kdc[3808](info): commencing operation
krb5kdc: starting...
W20260811 19:04:19.963290 13748 mini_kdc.cc:121] Time spent starting KDC: real 1.070s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474792496492-13748-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474792496492-13748-0/krb5kdc/test-user.keytab.
Aug 11 19:04:20 dist-test-slave-0kls krb5kdc[3808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475060, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:04:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:04:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474792496492-13748-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474792496492-13748-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 19:04:20.105423 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474792496492-13748-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474792496492-13748-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474792496492-13748-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46861
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474792496492-13748-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46861
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:04:20.227315 3822 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:04:20.227702 3822 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:04:20.227787 3822 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:04:20.232102 3822 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:04:20.232219 3822 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:04:20.232261 3822 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:04:20.232317 3822 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:04:20.232357 3822 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:04:20.238242 3822 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474792496492-13748-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474792496492-13748-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46861
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474792496492-13748-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46861
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3822
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474792496492-13748-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:04:20.239879 3822 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:04:20.240984 3822 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:04:20.250175 3828 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:04:20.250175 3827 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:04:20.250204 3830 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:04:20.250628 3822 server_base.cc:1034] running on GCE node
I20260811 19:04:20.251274 3822 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:04:20.252544 3822 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:04:20.253755 3822 hybrid_clock.cc:648] HybridClock initialized: now 1786475060253738 us; error 39 us; skew 500 ppm
Aug 11 19:04:20 dist-test-slave-0kls krb5kdc[3808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475060, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:04:20.265301 3822 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 19:04:20.266489 3822 webserver.cc:460] Webserver started at http://127.0.0.1:36571/ using document root <none> and password file <none>
I20260811 19:04:20.267122 3822 fs_manager.cc:359] Metadata directory not provided
I20260811 19:04:20.267220 3822 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:04:20.267482 3822 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:04:20.269824 3822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474792496492-13748-0/minicluster-data/master-0/data/instance:
uuid: "cfdc6d732117491d80868ed5086b3445"
format_stamp: "Formatted at 2026-08-11 19:04:20 on dist-test-slave-0kls"
server_key: "3eb655afb8a63f3ff93e3b4e6c0eab40"
server_key_iv: "1bbb22a9deb6f3a4d076c08f5ce981e8"
server_key_version: "encryptionkey@0"
I20260811 19:04:20.270466 3822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474792496492-13748-0/minicluster-data/master-0/wal/instance:
uuid: "cfdc6d732117491d80868ed5086b3445"
format_stamp: "Formatted at 2026-08-11 19:04:20 on dist-test-slave-0kls"
server_key: "3eb655afb8a63f3ff93e3b4e6c0eab40"
server_key_iv: "1bbb22a9deb6f3a4d076c08f5ce981e8"
server_key_version: "encryptionkey@0"
I20260811 19:04:20.274438 3822 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 19:04:20.277030 3838 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:04:20.278419 3822 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 19:04:20.278579 3822 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474792496492-13748-0/minicluster-data/master-0/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474792496492-13748-0/minicluster-data/master-0/wal
uuid: "cfdc6d732117491d80868ed5086b3445"
format_stamp: "Formatted at 2026-08-11 19:04:20 on dist-test-slave-0kls"
server_key: "3eb655afb8a63f3ff93e3b4e6c0eab40"
server_key_iv: "1bbb22a9deb6f3a4d076c08f5ce981e8"
server_key_version: "encryptionkey@0"
I20260811 19:04:20.278703 3822 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474792496492-13748-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474792496492-13748-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474792496492-13748-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:04:20.283433 3822 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:04:20.286626 3822 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:04:20.286849 3822 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:04:20.294565 3822 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46861
I20260811 19:04:20.294574 3896 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46861 every 8 connection(s)
I20260811 19:04:20.295615 3822 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
I20260811 19:04:20.298764 3897 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:04:20.304566 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 3822
I20260811 19:04:20.304752 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474792496492-13748-0/minicluster-data/master-0/wal/instance
I20260811 19:04:20.305037 13748 external_mini_cluster.cc:1428] Setting key 149c7f85928c1515d31411644624816a
I20260811 19:04:20.305109 3897 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfdc6d732117491d80868ed5086b3445: Bootstrap starting.
I20260811 19:04:20.308053 3897 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cfdc6d732117491d80868ed5086b3445: Neither blocks nor log segments found. Creating new log.
I20260811 19:04:20.309069 3897 log.cc:824] T 00000000000000000000000000000000 P cfdc6d732117491d80868ed5086b3445: Log is configured to *not* fsync() on all Append() calls
I20260811 19:04:20.311182 3900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:20.306697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34443 (local address 127.0.0.1:46861)
0811 19:04:20.307213 (+ 516us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:20.307229 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:20.307270 (+ 41us) server_negotiation.cc:407] Connection header received
0811 19:04:20.307981 (+ 711us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:20.308000 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:20.308379 (+ 379us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:20.308697 (+ 318us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:20.309703 (+ 1006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:04:20.309710 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:04:20.309727 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:04:20.309741 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:04:20.309844 (+ 103us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:04:20.309881 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:04:20.309998 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:20.310210 (+ 212us) server_negotiation.cc:299] Negotiation successful
0811 19:04:20.310401 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":162,"threads_started":1}
I20260811 19:04:20.311672 3897 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfdc6d732117491d80868ed5086b3445: No bootstrap required, opened a new log
I20260811 19:04:20.314476 13748 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 19:04:20.316095 3897 raft_consensus.cc:348] T 00000000000000000000000000000000 P cfdc6d732117491d80868ed5086b3445 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfdc6d732117491d80868ed5086b3445" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46861 } }
I20260811 19:04:20.316359 3897 raft_consensus.cc:374] T 00000000000000000000000000000000 P cfdc6d732117491d80868ed5086b3445 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:04:20.316421 3897 raft_consensus.cc:729] T 00000000000000000000000000000000 P cfdc6d732117491d80868ed5086b3445 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfdc6d732117491d80868ed5086b3445, State: Initialized, Role: FOLLOWER
I20260811 19:04:20.316985 3897 consensus_queue.cc:260] T 00000000000000000000000000000000 P cfdc6d732117491d80868ed5086b3445 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfdc6d732117491d80868ed5086b3445" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46861 } }
I20260811 19:04:20.317193 3897 raft_consensus.cc:388] T 00000000000000000000000000000000 P cfdc6d732117491d80868ed5086b3445 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:04:20.317284 3897 raft_consensus.cc:482] T 00000000000000000000000000000000 P cfdc6d732117491d80868ed5086b3445 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:04:20.317428 3897 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cfdc6d732117491d80868ed5086b3445 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:04:20.318689 3900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:20.316818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57025 (local address 127.0.0.1:46861)
0811 19:04:20.316965 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:20.316970 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:20.317256 (+ 286us) server_negotiation.cc:407] Connection header received
0811 19:04:20.317314 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:20.317320 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:20.317370 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:20.317463 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:20.318273 (+ 810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:04:20.318276 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:04:20.318278 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:04:20.318290 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:04:20.318319 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:04:20.318329 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:04:20.318406 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:20.318518 (+ 112us) server_negotiation.cc:299] Negotiation successful
0811 19:04:20.318552 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 19:04:20.318941 3897 raft_consensus.cc:504] T 00000000000000000000000000000000 P cfdc6d732117491d80868ed5086b3445 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfdc6d732117491d80868ed5086b3445" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46861 } }
I20260811 19:04:20.319341 3897 leader_election.cc:302] T 00000000000000000000000000000000 P cfdc6d732117491d80868ed5086b3445 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cfdc6d732117491d80868ed5086b3445; no voters:
I20260811 19:04:20.319733 3897 leader_election.cc:288] T 00000000000000000000000000000000 P cfdc6d732117491d80868ed5086b3445 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:04:20.319868 3909 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cfdc6d732117491d80868ed5086b3445 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:04:20.320092 3909 raft_consensus.cc:686] T 00000000000000000000000000000000 P cfdc6d732117491d80868ed5086b3445 [term 1 LEADER]: Becoming Leader. State: Replica: cfdc6d732117491d80868ed5086b3445, State: Running, Role: LEADER
I20260811 19:04:20.320441 3909 consensus_queue.cc:237] T 00000000000000000000000000000000 P cfdc6d732117491d80868ed5086b3445 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfdc6d732117491d80868ed5086b3445" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46861 } }
I20260811 19:04:20.321215 3897 sys_catalog.cc:564] T 00000000000000000000000000000000 P cfdc6d732117491d80868ed5086b3445 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:04:20.322599 3910 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfdc6d732117491d80868ed5086b3445 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cfdc6d732117491d80868ed5086b3445" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfdc6d732117491d80868ed5086b3445" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46861 } } }
I20260811 19:04:20.322628 3911 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfdc6d732117491d80868ed5086b3445 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cfdc6d732117491d80868ed5086b3445. Latest consensus state: current_term: 1 leader_uuid: "cfdc6d732117491d80868ed5086b3445" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfdc6d732117491d80868ed5086b3445" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46861 } } }
I20260811 19:04:20.322762 3911 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfdc6d732117491d80868ed5086b3445 [sys.catalog]: This master's current role is: LEADER
I20260811 19:04:20.322762 3910 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfdc6d732117491d80868ed5086b3445 [sys.catalog]: This master's current role is: LEADER
I20260811 19:04:20.323244 3918 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:04:20.326284 3918 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:04:20.332621 3918 catalog_manager.cc:1347] Generated new cluster ID: d1347cc0c0d342b78a64cd08f3a50eb5
I20260811 19:04:20.332720 3918 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:04:20.342693 3918 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:04:20.343093 3918 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:04:20.348542 3918 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cfdc6d732117491d80868ed5086b3445: Generated new TSK 0
I20260811 19:04:20.349344 3918 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 19:04:20.362792 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 3822
2026-08-11T19:04:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1492 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474792496492-13748-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:04:20 dist-test-slave-0kls krb5kdc[3930](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:04:20 dist-test-slave-0kls krb5kdc[3930](info): set up 2 sockets
Aug 11 19:04:20 dist-test-slave-0kls krb5kdc[3930](info): commencing operation
krb5kdc: starting...
W20260811 19:04:21.459014 13748 mini_kdc.cc:121] Time spent starting KDC: real 1.073s user 0.006s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474792496492-13748-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474792496492-13748-0/krb5kdc/test-user.keytab.
Aug 11 19:04:21 dist-test-slave-0kls krb5kdc[3930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475061, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:04:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:04:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474792496492-13748-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474792496492-13748-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 19:04:21.597370 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474792496492-13748-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474792496492-13748-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474792496492-13748-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46789
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474792496492-13748-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46789
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:04:21.721606 3947 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:04:21.721998 3947 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:04:21.722105 3947 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:04:21.726500 3947 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:04:21.726604 3947 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:04:21.726644 3947 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:04:21.726703 3947 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:04:21.726742 3947 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:04:21.732676 3947 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474792496492-13748-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474792496492-13748-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46789
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474792496492-13748-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46789
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3947
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474792496492-13748-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:04:21.733974 3947 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:04:21.735167 3947 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:04:21.744217 3953 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:04:21.744180 3954 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:04:21.744270 3956 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:04:21.744657 3947 server_base.cc:1034] running on GCE node
I20260811 19:04:21.745318 3947 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:04:21.746517 3947 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:04:21.747709 3947 hybrid_clock.cc:648] HybridClock initialized: now 1786475061747693 us; error 28 us; skew 500 ppm
Aug 11 19:04:21 dist-test-slave-0kls krb5kdc[3930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475061, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:04:21.750901 3947 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 19:04:21.752017 3947 webserver.cc:460] Webserver started at http://127.0.0.1:46073/ using document root <none> and password file <none>
I20260811 19:04:21.752652 3947 fs_manager.cc:359] Metadata directory not provided
I20260811 19:04:21.752753 3947 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:04:21.753007 3947 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:04:21.755417 3947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474792496492-13748-0/minicluster-data/master-0/data/instance:
uuid: "3011e015d43e4a9fb4c8f8ccec084c01"
format_stamp: "Formatted at 2026-08-11 19:04:21 on dist-test-slave-0kls"
server_key: "95f6aa5fbf02f4afe70feafd6314e108"
server_key_iv: "61672d1353d66e37f82ac5c680a3427d"
server_key_version: "encryptionkey@0"
I20260811 19:04:21.756021 3947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474792496492-13748-0/minicluster-data/master-0/wal/instance:
uuid: "3011e015d43e4a9fb4c8f8ccec084c01"
format_stamp: "Formatted at 2026-08-11 19:04:21 on dist-test-slave-0kls"
server_key: "95f6aa5fbf02f4afe70feafd6314e108"
server_key_iv: "61672d1353d66e37f82ac5c680a3427d"
server_key_version: "encryptionkey@0"
I20260811 19:04:21.760371 3947 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 19:04:21.763186 3963 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:04:21.764606 3947 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 19:04:21.764775 3947 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474792496492-13748-0/minicluster-data/master-0/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474792496492-13748-0/minicluster-data/master-0/wal
uuid: "3011e015d43e4a9fb4c8f8ccec084c01"
format_stamp: "Formatted at 2026-08-11 19:04:21 on dist-test-slave-0kls"
server_key: "95f6aa5fbf02f4afe70feafd6314e108"
server_key_iv: "61672d1353d66e37f82ac5c680a3427d"
server_key_version: "encryptionkey@0"
I20260811 19:04:21.764909 3947 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474792496492-13748-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474792496492-13748-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474792496492-13748-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:04:21.778867 3947 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:04:21.782491 3947 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:04:21.782737 3947 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:04:21.790532 3947 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46789
I20260811 19:04:21.790563 4024 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46789 every 8 connection(s)
I20260811 19:04:21.791582 3947 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
I20260811 19:04:21.794545 4025 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:04:21.796001 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 3947
I20260811 19:04:21.796167 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474792496492-13748-0/minicluster-data/master-0/wal/instance
I20260811 19:04:21.796514 13748 external_mini_cluster.cc:1428] Setting key bfdc80759528de85cd25c0d7493ecb22
I20260811 19:04:21.801069 4025 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3011e015d43e4a9fb4c8f8ccec084c01: Bootstrap starting.
I20260811 19:04:21.803463 4025 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3011e015d43e4a9fb4c8f8ccec084c01: Neither blocks nor log segments found. Creating new log.
I20260811 19:04:21.804244 4025 log.cc:824] T 00000000000000000000000000000000 P 3011e015d43e4a9fb4c8f8ccec084c01: Log is configured to *not* fsync() on all Append() calls
Aug 11 19:04:21 dist-test-slave-0kls krb5kdc[3930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475061, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 19:04:21.806479 4025 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3011e015d43e4a9fb4c8f8ccec084c01: No bootstrap required, opened a new log
I20260811 19:04:21.810050 4025 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3011e015d43e4a9fb4c8f8ccec084c01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3011e015d43e4a9fb4c8f8ccec084c01" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46789 } }
I20260811 19:04:21.810325 4025 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3011e015d43e4a9fb4c8f8ccec084c01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:04:21.810395 4025 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3011e015d43e4a9fb4c8f8ccec084c01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3011e015d43e4a9fb4c8f8ccec084c01, State: Initialized, Role: FOLLOWER
I20260811 19:04:21.810868 4025 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3011e015d43e4a9fb4c8f8ccec084c01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3011e015d43e4a9fb4c8f8ccec084c01" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46789 } }
I20260811 19:04:21.811033 4025 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3011e015d43e4a9fb4c8f8ccec084c01 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:04:21.811164 4025 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3011e015d43e4a9fb4c8f8ccec084c01 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:04:21.811296 4025 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3011e015d43e4a9fb4c8f8ccec084c01 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:04:21.811548 4028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:21.798187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44399 (local address 127.0.0.1:46789)
0811 19:04:21.798658 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:21.798674 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:21.798712 (+ 38us) server_negotiation.cc:407] Connection header received
0811 19:04:21.799351 (+ 639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:21.799367 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:21.799704 (+ 337us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:21.800087 (+ 383us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:21.800895 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:21.802018 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:21.802972 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:21.803459 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:21.803640 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:21.806489 (+ 2849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:04:21.806494 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:04:21.806509 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:04:21.806538 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:04:21.808349 (+ 1811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:21.808821 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:21.808827 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:21.808834 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:21.808957 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:21.809252 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:21.809255 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:21.809257 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:21.809680 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:04:21.809921 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:21.810257 (+ 336us) server_negotiation.cc:299] Negotiation successful
0811 19:04:21.810598 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":144,"threads_started":1}
I20260811 19:04:21.812546 4025 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3011e015d43e4a9fb4c8f8ccec084c01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3011e015d43e4a9fb4c8f8ccec084c01" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46789 } }
I20260811 19:04:21.812908 4025 leader_election.cc:302] T 00000000000000000000000000000000 P 3011e015d43e4a9fb4c8f8ccec084c01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3011e015d43e4a9fb4c8f8ccec084c01; no voters:
I20260811 19:04:21.813267 4025 leader_election.cc:288] T 00000000000000000000000000000000 P 3011e015d43e4a9fb4c8f8ccec084c01 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:04:21.813359 4030 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3011e015d43e4a9fb4c8f8ccec084c01 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:04:21.813539 4030 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3011e015d43e4a9fb4c8f8ccec084c01 [term 1 LEADER]: Becoming Leader. State: Replica: 3011e015d43e4a9fb4c8f8ccec084c01, State: Running, Role: LEADER
I20260811 19:04:21.813870 4030 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3011e015d43e4a9fb4c8f8ccec084c01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3011e015d43e4a9fb4c8f8ccec084c01" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46789 } }
I20260811 19:04:21.814627 4025 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3011e015d43e4a9fb4c8f8ccec084c01 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:04:21.814939 13748 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 19:04:21.815639 4032 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3011e015d43e4a9fb4c8f8ccec084c01 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3011e015d43e4a9fb4c8f8ccec084c01. Latest consensus state: current_term: 1 leader_uuid: "3011e015d43e4a9fb4c8f8ccec084c01" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3011e015d43e4a9fb4c8f8ccec084c01" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46789 } } }
I20260811 19:04:21.815763 4032 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3011e015d43e4a9fb4c8f8ccec084c01 [sys.catalog]: This master's current role is: LEADER
I20260811 19:04:21.816054 4031 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3011e015d43e4a9fb4c8f8ccec084c01 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3011e015d43e4a9fb4c8f8ccec084c01" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3011e015d43e4a9fb4c8f8ccec084c01" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46789 } } }
I20260811 19:04:21.816140 4031 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3011e015d43e4a9fb4c8f8ccec084c01 [sys.catalog]: This master's current role is: LEADER
I20260811 19:04:21.816186 4040 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:04:21.819531 4040 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:04:21.826819 4040 catalog_manager.cc:1347] Generated new cluster ID: d332274455d043978443324c4b9023fb
I20260811 19:04:21.826920 4040 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:04:21.826980 4028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:21.817707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37075 (local address 127.0.0.1:46789)
0811 19:04:21.817835 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:21.817840 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:21.817854 (+ 14us) server_negotiation.cc:407] Connection header received
0811 19:04:21.817966 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:21.817969 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:21.818011 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:21.818101 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:21.819455 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:21.820083 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:21.821212 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:21.821462 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:21.821578 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:21.823255 (+ 1677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:04:21.823259 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:04:21.823263 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:04:21.823290 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:04:21.825073 (+ 1783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:21.825511 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:21.825515 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:21.825517 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:21.825609 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:21.825959 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:21.825963 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:21.825965 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:21.826163 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:04:21.826299 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:21.826620 (+ 321us) server_negotiation.cc:299] Negotiation successful
0811 19:04:21.826743 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260811 19:04:21.835268 4040 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:04:21.836210 4040 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:04:21.844120 4040 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3011e015d43e4a9fb4c8f8ccec084c01: Generated new TSK 0
I20260811 19:04:21.844820 4040 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 19:04:21.854913 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 3947
2026-08-11T19:04:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1492 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474792496492-13748-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:04:21 dist-test-slave-0kls krb5kdc[4058](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:04:21 dist-test-slave-0kls krb5kdc[4058](info): set up 2 sockets
Aug 11 19:04:21 dist-test-slave-0kls krb5kdc[4058](info): commencing operation
krb5kdc: starting...
W20260811 19:04:22.955885 13748 mini_kdc.cc:121] Time spent starting KDC: real 1.078s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474792496492-13748-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474792496492-13748-0/krb5kdc/test-user.keytab.
Aug 11 19:04:23 dist-test-slave-0kls krb5kdc[4058](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475063, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:04:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:04:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474792496492-13748-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474792496492-13748-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 19:04:23.093429 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474792496492-13748-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474792496492-13748-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474792496492-13748-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37095
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474792496492-13748-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37095
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:04:23.211406 4073 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:04:23.211716 4073 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:04:23.211762 4073 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:04:23.215874 4073 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:04:23.215950 4073 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:04:23.215970 4073 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:04:23.215987 4073 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:04:23.216003 4073 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:04:23.221709 4073 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474792496492-13748-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474792496492-13748-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37095
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474792496492-13748-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37095
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4073
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474792496492-13748-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:04:23.223031 4073 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:04:23.224171 4073 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:04:23.233644 4079 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:04:23.233626 4082 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:04:23.233747 4073 server_base.cc:1034] running on GCE node
W20260811 19:04:23.233626 4080 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:04:23.234709 4073 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:04:23.235970 4073 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:04:23.237185 4073 hybrid_clock.cc:648] HybridClock initialized: now 1786475063237154 us; error 45 us; skew 500 ppm
Aug 11 19:04:23 dist-test-slave-0kls krb5kdc[4058](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475063, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:04:23.240247 4073 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 19:04:23.241294 4073 webserver.cc:460] Webserver started at http://127.0.0.1:43629/ using document root <none> and password file <none>
I20260811 19:04:23.241958 4073 fs_manager.cc:359] Metadata directory not provided
I20260811 19:04:23.242067 4073 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:04:23.242321 4073 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:04:23.244491 4073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474792496492-13748-0/minicluster-data/master-0/data/instance:
uuid: "052e59572d5849d0882c0b012ca42310"
format_stamp: "Formatted at 2026-08-11 19:04:23 on dist-test-slave-0kls"
server_key: "2baa63f1dedf2b8ba44fdfbd3ee7c1e9"
server_key_iv: "b6607c00fa3fbf5db6c14555f63847cc"
server_key_version: "encryptionkey@0"
I20260811 19:04:23.245106 4073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474792496492-13748-0/minicluster-data/master-0/wal/instance:
uuid: "052e59572d5849d0882c0b012ca42310"
format_stamp: "Formatted at 2026-08-11 19:04:23 on dist-test-slave-0kls"
server_key: "2baa63f1dedf2b8ba44fdfbd3ee7c1e9"
server_key_iv: "b6607c00fa3fbf5db6c14555f63847cc"
server_key_version: "encryptionkey@0"
I20260811 19:04:23.249121 4073 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260811 19:04:23.251804 4091 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:04:23.253250 4073 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 19:04:23.253408 4073 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474792496492-13748-0/minicluster-data/master-0/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474792496492-13748-0/minicluster-data/master-0/wal
uuid: "052e59572d5849d0882c0b012ca42310"
format_stamp: "Formatted at 2026-08-11 19:04:23 on dist-test-slave-0kls"
server_key: "2baa63f1dedf2b8ba44fdfbd3ee7c1e9"
server_key_iv: "b6607c00fa3fbf5db6c14555f63847cc"
server_key_version: "encryptionkey@0"
I20260811 19:04:23.253535 4073 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474792496492-13748-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474792496492-13748-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474792496492-13748-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:04:23.262863 4073 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:04:23.266247 4073 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:04:23.266458 4073 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:04:23.274360 4073 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37095
I20260811 19:04:23.274384 4151 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37095 every 8 connection(s)
I20260811 19:04:23.275394 4073 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
I20260811 19:04:23.278311 4152 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:04:23.281343 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 4073
I20260811 19:04:23.281504 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474792496492-13748-0/minicluster-data/master-0/wal/instance
I20260811 19:04:23.281852 13748 external_mini_cluster.cc:1428] Setting key 018049dbf4f501a18e65f59714cdebc3
I20260811 19:04:23.284590 4152 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 052e59572d5849d0882c0b012ca42310: Bootstrap starting.
I20260811 19:04:23.286964 4152 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 052e59572d5849d0882c0b012ca42310: Neither blocks nor log segments found. Creating new log.
I20260811 19:04:23.287741 4152 log.cc:824] T 00000000000000000000000000000000 P 052e59572d5849d0882c0b012ca42310: Log is configured to *not* fsync() on all Append() calls
I20260811 19:04:23.289741 4152 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 052e59572d5849d0882c0b012ca42310: No bootstrap required, opened a new log
I20260811 19:04:23.291159 4156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:23.283367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36781 (local address 127.0.0.1:37095)
0811 19:04:23.283825 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:23.283841 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:23.283876 (+ 35us) server_negotiation.cc:407] Connection header received
0811 19:04:23.284529 (+ 653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:23.284547 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:23.284933 (+ 386us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:23.285271 (+ 338us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:23.285691 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:23.286739 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:23.287709 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:23.288230 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:23.288405 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:23.289298 (+ 893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:04:23.289303 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:04:23.289317 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:04:23.289700 (+ 383us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:04:23.289775 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:04:23.289846 (+ 71us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:04:23.289953 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:23.290166 (+ 213us) server_negotiation.cc:299] Negotiation successful
0811 19:04:23.290405 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":146,"threads_started":1}
I20260811 19:04:23.293627 4152 raft_consensus.cc:348] T 00000000000000000000000000000000 P 052e59572d5849d0882c0b012ca42310 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "052e59572d5849d0882c0b012ca42310" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37095 } }
I20260811 19:04:23.293946 4152 raft_consensus.cc:374] T 00000000000000000000000000000000 P 052e59572d5849d0882c0b012ca42310 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:04:23.294063 4152 raft_consensus.cc:729] T 00000000000000000000000000000000 P 052e59572d5849d0882c0b012ca42310 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 052e59572d5849d0882c0b012ca42310, State: Initialized, Role: FOLLOWER
I20260811 19:04:23.294122 13748 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 19:04:23.294565 4152 consensus_queue.cc:260] T 00000000000000000000000000000000 P 052e59572d5849d0882c0b012ca42310 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "052e59572d5849d0882c0b012ca42310" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37095 } }
I20260811 19:04:23.294732 4152 raft_consensus.cc:388] T 00000000000000000000000000000000 P 052e59572d5849d0882c0b012ca42310 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:04:23.294824 4152 raft_consensus.cc:482] T 00000000000000000000000000000000 P 052e59572d5849d0882c0b012ca42310 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:04:23.294926 4152 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 052e59572d5849d0882c0b012ca42310 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:04:23.296020 4152 raft_consensus.cc:504] T 00000000000000000000000000000000 P 052e59572d5849d0882c0b012ca42310 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "052e59572d5849d0882c0b012ca42310" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37095 } }
I20260811 19:04:23.296458 4152 leader_election.cc:302] T 00000000000000000000000000000000 P 052e59572d5849d0882c0b012ca42310 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 052e59572d5849d0882c0b012ca42310; no voters:
I20260811 19:04:23.296865 4152 leader_election.cc:288] T 00000000000000000000000000000000 P 052e59572d5849d0882c0b012ca42310 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:04:23.297008 4163 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 052e59572d5849d0882c0b012ca42310 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:04:23.297283 4163 raft_consensus.cc:686] T 00000000000000000000000000000000 P 052e59572d5849d0882c0b012ca42310 [term 1 LEADER]: Becoming Leader. State: Replica: 052e59572d5849d0882c0b012ca42310, State: Running, Role: LEADER
I20260811 19:04:23.297703 4163 consensus_queue.cc:237] T 00000000000000000000000000000000 P 052e59572d5849d0882c0b012ca42310 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "052e59572d5849d0882c0b012ca42310" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37095 } }
I20260811 19:04:23.298434 4152 sys_catalog.cc:564] T 00000000000000000000000000000000 P 052e59572d5849d0882c0b012ca42310 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:04:23.299635 4166 sys_catalog.cc:455] T 00000000000000000000000000000000 P 052e59572d5849d0882c0b012ca42310 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 052e59572d5849d0882c0b012ca42310. Latest consensus state: current_term: 1 leader_uuid: "052e59572d5849d0882c0b012ca42310" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "052e59572d5849d0882c0b012ca42310" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37095 } } }
I20260811 19:04:23.299804 4166 sys_catalog.cc:458] T 00000000000000000000000000000000 P 052e59572d5849d0882c0b012ca42310 [sys.catalog]: This master's current role is: LEADER
I20260811 19:04:23.300067 4164 sys_catalog.cc:455] T 00000000000000000000000000000000 P 052e59572d5849d0882c0b012ca42310 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "052e59572d5849d0882c0b012ca42310" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "052e59572d5849d0882c0b012ca42310" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37095 } } }
I20260811 19:04:23.300170 4164 sys_catalog.cc:458] T 00000000000000000000000000000000 P 052e59572d5849d0882c0b012ca42310 [sys.catalog]: This master's current role is: LEADER
I20260811 19:04:23.301520 4156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:23.296379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40397 (local address 127.0.0.1:37095)
0811 19:04:23.296533 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:23.296538 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:23.296691 (+ 153us) server_negotiation.cc:407] Connection header received
0811 19:04:23.296836 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:23.296839 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:23.296899 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:23.296998 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:23.297479 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:23.298257 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:23.299511 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:23.299869 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:23.299979 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:23.300974 (+ 995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:04:23.300978 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:04:23.300980 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:04:23.300996 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:04:23.301029 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:04:23.301044 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:04:23.301093 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:23.301296 (+ 203us) server_negotiation.cc:299] Negotiation successful
0811 19:04:23.301370 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
W20260811 19:04:23.303929 4181 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 052e59572d5849d0882c0b012ca42310: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 19:04:23.304031 4181 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 19:04:23.304108 4173 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:04:23.304858 4173 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:04:23.311309 4173 catalog_manager.cc:1347] Generated new cluster ID: 35c6fbe9be6e41989aa007e5ea7090e1
I20260811 19:04:23.311394 4173 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:04:23.329927 4173 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:04:23.331075 4173 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:04:23.337522 4173 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 052e59572d5849d0882c0b012ca42310: Generated new TSK 0
I20260811 19:04:23.338446 4173 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 19:04:23.344640 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 4073
2026-08-11T19:04:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1490 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474792496492-13748-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:04:23 dist-test-slave-0kls krb5kdc[4188](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:04:23 dist-test-slave-0kls krb5kdc[4188](info): set up 2 sockets
Aug 11 19:04:23 dist-test-slave-0kls krb5kdc[4188](info): commencing operation
krb5kdc: starting...
W20260811 19:04:24.448592 13748 mini_kdc.cc:121] Time spent starting KDC: real 1.080s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474792496492-13748-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474792496492-13748-0/krb5kdc/test-user.keytab.
Aug 11 19:04:24 dist-test-slave-0kls krb5kdc[4188](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475064, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:04:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:04:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474792496492-13748-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474792496492-13748-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 19:04:24.589593 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474792496492-13748-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474792496492-13748-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474792496492-13748-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33717
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474792496492-13748-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33717
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:04:24.711584 4201 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:04:24.711975 4201 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:04:24.712066 4201 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:04:24.716444 4201 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:04:24.716558 4201 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:04:24.716596 4201 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:04:24.716624 4201 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:04:24.716661 4201 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:04:24.722508 4201 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474792496492-13748-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474792496492-13748-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33717
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474792496492-13748-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33717
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4201
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474792496492-13748-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:04:24.723827 4201 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:04:24.724978 4201 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:04:24.734217 4210 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:04:24.734268 4201 server_base.cc:1034] running on GCE node
W20260811 19:04:24.734297 4207 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:04:24.734244 4206 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:04:24.735074 4201 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:04:24.736522 4201 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:04:24.737707 4201 hybrid_clock.cc:648] HybridClock initialized: now 1786475064737665 us; error 58 us; skew 500 ppm
Aug 11 19:04:24 dist-test-slave-0kls krb5kdc[4188](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475064, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:04:24.740918 4201 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 19:04:24.741997 4201 webserver.cc:460] Webserver started at http://127.0.0.1:37033/ using document root <none> and password file <none>
I20260811 19:04:24.742604 4201 fs_manager.cc:359] Metadata directory not provided
I20260811 19:04:24.742699 4201 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:04:24.742950 4201 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:04:24.745110 4201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474792496492-13748-0/minicluster-data/master-0/data/instance:
uuid: "89e855c60ad245ccab8a4d0c33902922"
format_stamp: "Formatted at 2026-08-11 19:04:24 on dist-test-slave-0kls"
server_key: "80588b25c0fdeefa4cd5fb2fa1a5e083"
server_key_iv: "ea8dd35d484485a81c4247c5d7176555"
server_key_version: "encryptionkey@0"
I20260811 19:04:24.745711 4201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474792496492-13748-0/minicluster-data/master-0/wal/instance:
uuid: "89e855c60ad245ccab8a4d0c33902922"
format_stamp: "Formatted at 2026-08-11 19:04:24 on dist-test-slave-0kls"
server_key: "80588b25c0fdeefa4cd5fb2fa1a5e083"
server_key_iv: "ea8dd35d484485a81c4247c5d7176555"
server_key_version: "encryptionkey@0"
I20260811 19:04:24.749717 4201 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260811 19:04:24.752429 4219 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:04:24.754041 4201 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 19:04:24.754179 4201 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474792496492-13748-0/minicluster-data/master-0/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474792496492-13748-0/minicluster-data/master-0/wal
uuid: "89e855c60ad245ccab8a4d0c33902922"
format_stamp: "Formatted at 2026-08-11 19:04:24 on dist-test-slave-0kls"
server_key: "80588b25c0fdeefa4cd5fb2fa1a5e083"
server_key_iv: "ea8dd35d484485a81c4247c5d7176555"
server_key_version: "encryptionkey@0"
I20260811 19:04:24.754280 4201 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474792496492-13748-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474792496492-13748-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474792496492-13748-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:04:24.759114 4201 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:04:24.762458 4201 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:04:24.762614 4201 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:04:24.769790 4201 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33717
I20260811 19:04:24.769886 4277 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33717 every 8 connection(s)
I20260811 19:04:24.770848 4201 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
I20260811 19:04:24.774101 4278 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:04:24.778013 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 4201
I20260811 19:04:24.778183 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474792496492-13748-0/minicluster-data/master-0/wal/instance
I20260811 19:04:24.778465 13748 external_mini_cluster.cc:1428] Setting key aa72a10fead7c4d066ffd1058b8fcaa9
I20260811 19:04:24.780496 4278 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89e855c60ad245ccab8a4d0c33902922: Bootstrap starting.
I20260811 19:04:24.783583 4278 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 89e855c60ad245ccab8a4d0c33902922: Neither blocks nor log segments found. Creating new log.
I20260811 19:04:24.784646 4278 log.cc:824] T 00000000000000000000000000000000 P 89e855c60ad245ccab8a4d0c33902922: Log is configured to *not* fsync() on all Append() calls
I20260811 19:04:24.784708 4282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:24.779988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45807 (local address 127.0.0.1:33717)
0811 19:04:24.780480 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:24.780501 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:24.780562 (+ 61us) server_negotiation.cc:407] Connection header received
0811 19:04:24.781418 (+ 856us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:24.781442 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:24.781908 (+ 466us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:24.782310 (+ 402us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:24.783116 (+ 806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:04:24.783122 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:04:24.783137 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:04:24.783150 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:04:24.783219 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:04:24.783246 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:04:24.783364 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:24.783548 (+ 184us) server_negotiation.cc:299] Negotiation successful
0811 19:04:24.783725 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":115,"threads_started":1}
I20260811 19:04:24.787144 4278 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89e855c60ad245ccab8a4d0c33902922: No bootstrap required, opened a new log
I20260811 19:04:24.788103 13748 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 19:04:24.791704 4278 raft_consensus.cc:348] T 00000000000000000000000000000000 P 89e855c60ad245ccab8a4d0c33902922 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89e855c60ad245ccab8a4d0c33902922" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33717 } }
I20260811 19:04:24.792075 4278 raft_consensus.cc:374] T 00000000000000000000000000000000 P 89e855c60ad245ccab8a4d0c33902922 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:04:24.792157 4278 raft_consensus.cc:729] T 00000000000000000000000000000000 P 89e855c60ad245ccab8a4d0c33902922 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89e855c60ad245ccab8a4d0c33902922, State: Initialized, Role: FOLLOWER
I20260811 19:04:24.792357 4282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:24.790516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50389 (local address 127.0.0.1:33717)
0811 19:04:24.790683 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:24.790689 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:24.790855 (+ 166us) server_negotiation.cc:407] Connection header received
0811 19:04:24.790981 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:24.790983 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:24.791025 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:24.791100 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:24.791893 (+ 793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:04:24.791896 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:04:24.791898 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:04:24.791910 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:04:24.791940 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:04:24.791952 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:04:24.792057 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:24.792187 (+ 130us) server_negotiation.cc:299] Negotiation successful
0811 19:04:24.792240 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 19:04:24.792714 4278 consensus_queue.cc:260] T 00000000000000000000000000000000 P 89e855c60ad245ccab8a4d0c33902922 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89e855c60ad245ccab8a4d0c33902922" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33717 } }
I20260811 19:04:24.792924 4278 raft_consensus.cc:388] T 00000000000000000000000000000000 P 89e855c60ad245ccab8a4d0c33902922 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:04:24.793011 4278 raft_consensus.cc:482] T 00000000000000000000000000000000 P 89e855c60ad245ccab8a4d0c33902922 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:04:24.793155 4278 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 89e855c60ad245ccab8a4d0c33902922 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:04:24.794379 4278 raft_consensus.cc:504] T 00000000000000000000000000000000 P 89e855c60ad245ccab8a4d0c33902922 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89e855c60ad245ccab8a4d0c33902922" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33717 } }
I20260811 19:04:24.794766 4278 leader_election.cc:302] T 00000000000000000000000000000000 P 89e855c60ad245ccab8a4d0c33902922 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 89e855c60ad245ccab8a4d0c33902922; no voters:
I20260811 19:04:24.795115 4278 leader_election.cc:288] T 00000000000000000000000000000000 P 89e855c60ad245ccab8a4d0c33902922 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:04:24.795284 4289 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 89e855c60ad245ccab8a4d0c33902922 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:04:24.795575 4289 raft_consensus.cc:686] T 00000000000000000000000000000000 P 89e855c60ad245ccab8a4d0c33902922 [term 1 LEADER]: Becoming Leader. State: Replica: 89e855c60ad245ccab8a4d0c33902922, State: Running, Role: LEADER
I20260811 19:04:24.795971 4289 consensus_queue.cc:237] T 00000000000000000000000000000000 P 89e855c60ad245ccab8a4d0c33902922 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89e855c60ad245ccab8a4d0c33902922" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33717 } }
I20260811 19:04:24.796542 4278 sys_catalog.cc:564] T 00000000000000000000000000000000 P 89e855c60ad245ccab8a4d0c33902922 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:04:24.798120 4292 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89e855c60ad245ccab8a4d0c33902922 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 89e855c60ad245ccab8a4d0c33902922. Latest consensus state: current_term: 1 leader_uuid: "89e855c60ad245ccab8a4d0c33902922" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89e855c60ad245ccab8a4d0c33902922" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33717 } } }
I20260811 19:04:24.798137 4290 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89e855c60ad245ccab8a4d0c33902922 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "89e855c60ad245ccab8a4d0c33902922" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89e855c60ad245ccab8a4d0c33902922" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33717 } } }
I20260811 19:04:24.798295 4292 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89e855c60ad245ccab8a4d0c33902922 [sys.catalog]: This master's current role is: LEADER
I20260811 19:04:24.798295 4290 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89e855c60ad245ccab8a4d0c33902922 [sys.catalog]: This master's current role is: LEADER
W20260811 19:04:24.801509 4310 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 89e855c60ad245ccab8a4d0c33902922: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 19:04:24.801630 4310 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 19:04:24.801754 4306 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:04:24.802850 4306 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:04:24.809144 4306 catalog_manager.cc:1347] Generated new cluster ID: 1ec2c6bfcf544445aee78fd9bcaddb7c
I20260811 19:04:24.809234 4306 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:04:24.833169 4306 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:04:24.833530 4306 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:04:24.846320 4306 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 89e855c60ad245ccab8a4d0c33902922: Generated new TSK 0
I20260811 19:04:24.847313 4306 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 19:04:24.853618 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 4201
2026-08-11T19:04:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1509 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474792496492-13748-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:04:24 dist-test-slave-0kls krb5kdc[4317](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:04:24 dist-test-slave-0kls krb5kdc[4317](info): set up 2 sockets
Aug 11 19:04:24 dist-test-slave-0kls krb5kdc[4317](info): commencing operation
krb5kdc: starting...
W20260811 19:04:25.948452 13748 mini_kdc.cc:121] Time spent starting KDC: real 1.070s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474792496492-13748-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474792496492-13748-0/krb5kdc/test-user.keytab.
Aug 11 19:04:26 dist-test-slave-0kls krb5kdc[4317](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475066, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:04:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:04:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474792496492-13748-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474792496492-13748-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 19:04:26.096936 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474792496492-13748-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474792496492-13748-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474792496492-13748-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35931
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474792496492-13748-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35931
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:04:26.236013 4332 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:04:26.236400 4332 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:04:26.236495 4332 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:04:26.241042 4332 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:04:26.241168 4332 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:04:26.241209 4332 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:04:26.241248 4332 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:04:26.241284 4332 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:04:26.247527 4332 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474792496492-13748-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474792496492-13748-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35931
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474792496492-13748-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35931
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4332
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474792496492-13748-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:04:26.248793 4332 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:04:26.249960 4332 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:04:26.259088 4337 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:04:26.259109 4341 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:04:26.259214 4339 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:04:26.259531 4332 server_base.cc:1034] running on GCE node
I20260811 19:04:26.260124 4332 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:04:26.261340 4332 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:04:26.262547 4332 hybrid_clock.cc:648] HybridClock initialized: now 1786475066262522 us; error 55 us; skew 500 ppm
Aug 11 19:04:26 dist-test-slave-0kls krb5kdc[4317](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475066, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:04:26.265568 4332 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 19:04:26.266671 4332 webserver.cc:460] Webserver started at http://127.0.0.1:37121/ using document root <none> and password file <none>
I20260811 19:04:26.267324 4332 fs_manager.cc:359] Metadata directory not provided
I20260811 19:04:26.267418 4332 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:04:26.267674 4332 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:04:26.269915 4332 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474792496492-13748-0/minicluster-data/master-0/data/instance:
uuid: "b2624067b0904345bbe2a05aa990e308"
format_stamp: "Formatted at 2026-08-11 19:04:26 on dist-test-slave-0kls"
server_key: "81dac013cdbabb760a32bb4ec4d6e7e4"
server_key_iv: "64dc2652fd271f094f703bd9a727d995"
server_key_version: "encryptionkey@0"
I20260811 19:04:26.270545 4332 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474792496492-13748-0/minicluster-data/master-0/wal/instance:
uuid: "b2624067b0904345bbe2a05aa990e308"
format_stamp: "Formatted at 2026-08-11 19:04:26 on dist-test-slave-0kls"
server_key: "81dac013cdbabb760a32bb4ec4d6e7e4"
server_key_iv: "64dc2652fd271f094f703bd9a727d995"
server_key_version: "encryptionkey@0"
I20260811 19:04:26.274690 4332 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 19:04:26.277359 4349 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:04:26.278785 4332 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 19:04:26.278940 4332 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474792496492-13748-0/minicluster-data/master-0/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474792496492-13748-0/minicluster-data/master-0/wal
uuid: "b2624067b0904345bbe2a05aa990e308"
format_stamp: "Formatted at 2026-08-11 19:04:26 on dist-test-slave-0kls"
server_key: "81dac013cdbabb760a32bb4ec4d6e7e4"
server_key_iv: "64dc2652fd271f094f703bd9a727d995"
server_key_version: "encryptionkey@0"
I20260811 19:04:26.279074 4332 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474792496492-13748-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474792496492-13748-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474792496492-13748-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:04:26.309574 4332 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:04:26.313374 4332 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:04:26.313633 4332 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:04:26.322126 4332 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35931
I20260811 19:04:26.322149 4410 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35931 every 8 connection(s)
I20260811 19:04:26.323339 4332 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
I20260811 19:04:26.325989 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 4332
I20260811 19:04:26.326172 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474792496492-13748-0/minicluster-data/master-0/wal/instance
I20260811 19:04:26.326519 13748 external_mini_cluster.cc:1428] Setting key abf0ea39e790915c20189164eefccdce
I20260811 19:04:26.327075 4411 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:04:26.334839 4411 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2624067b0904345bbe2a05aa990e308: Bootstrap starting.
Aug 11 19:04:26 dist-test-slave-0kls krb5kdc[4317](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475066, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 19:04:26.337982 4411 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b2624067b0904345bbe2a05aa990e308: Neither blocks nor log segments found. Creating new log.
I20260811 19:04:26.339053 4411 log.cc:824] T 00000000000000000000000000000000 P b2624067b0904345bbe2a05aa990e308: Log is configured to *not* fsync() on all Append() calls
I20260811 19:04:26.341928 4411 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2624067b0904345bbe2a05aa990e308: No bootstrap required, opened a new log
I20260811 19:04:26.343245 4415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:26.328138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.104:34645 (local address 127.0.0.1:35931)
0811 19:04:26.328639 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:26.328657 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:26.328698 (+ 41us) server_negotiation.cc:407] Connection header received
0811 19:04:26.329412 (+ 714us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:26.329432 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:26.329920 (+ 488us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:26.330257 (+ 337us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:26.331270 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:26.332291 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:26.333629 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:26.334246 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:26.334375 (+ 129us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:26.337371 (+ 2996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:04:26.337379 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:04:26.337398 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:04:26.337438 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:04:26.339612 (+ 2174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:26.340346 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:26.340353 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:26.340360 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:26.340494 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:26.340822 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:26.340825 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:26.340826 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:26.341371 (+ 545us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:04:26.341574 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:26.341958 (+ 384us) server_negotiation.cc:299] Negotiation successful
0811 19:04:26.342243 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":173,"threads_started":1}
I20260811 19:04:26.345726 4411 raft_consensus.cc:348] T 00000000000000000000000000000000 P b2624067b0904345bbe2a05aa990e308 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2624067b0904345bbe2a05aa990e308" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35931 } }
I20260811 19:04:26.346011 4411 raft_consensus.cc:374] T 00000000000000000000000000000000 P b2624067b0904345bbe2a05aa990e308 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:04:26.346093 4411 raft_consensus.cc:729] T 00000000000000000000000000000000 P b2624067b0904345bbe2a05aa990e308 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2624067b0904345bbe2a05aa990e308, State: Initialized, Role: FOLLOWER
I20260811 19:04:26.346580 4411 consensus_queue.cc:260] T 00000000000000000000000000000000 P b2624067b0904345bbe2a05aa990e308 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2624067b0904345bbe2a05aa990e308" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35931 } }
I20260811 19:04:26.346753 4411 raft_consensus.cc:388] T 00000000000000000000000000000000 P b2624067b0904345bbe2a05aa990e308 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:04:26.346861 4411 raft_consensus.cc:482] T 00000000000000000000000000000000 P b2624067b0904345bbe2a05aa990e308 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:04:26.346917 13748 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 19:04:26.346990 4411 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b2624067b0904345bbe2a05aa990e308 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:04:26.348014 4411 raft_consensus.cc:504] T 00000000000000000000000000000000 P b2624067b0904345bbe2a05aa990e308 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2624067b0904345bbe2a05aa990e308" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35931 } }
I20260811 19:04:26.348402 4411 leader_election.cc:302] T 00000000000000000000000000000000 P b2624067b0904345bbe2a05aa990e308 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b2624067b0904345bbe2a05aa990e308; no voters:
I20260811 19:04:26.348770 4411 leader_election.cc:288] T 00000000000000000000000000000000 P b2624067b0904345bbe2a05aa990e308 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:04:26.348850 4422 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b2624067b0904345bbe2a05aa990e308 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:04:26.349063 4422 raft_consensus.cc:686] T 00000000000000000000000000000000 P b2624067b0904345bbe2a05aa990e308 [term 1 LEADER]: Becoming Leader. State: Replica: b2624067b0904345bbe2a05aa990e308, State: Running, Role: LEADER
I20260811 19:04:26.349421 4422 consensus_queue.cc:237] T 00000000000000000000000000000000 P b2624067b0904345bbe2a05aa990e308 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2624067b0904345bbe2a05aa990e308" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35931 } }
I20260811 19:04:26.350205 4411 sys_catalog.cc:564] T 00000000000000000000000000000000 P b2624067b0904345bbe2a05aa990e308 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:04:26.351171 4424 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2624067b0904345bbe2a05aa990e308 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b2624067b0904345bbe2a05aa990e308" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2624067b0904345bbe2a05aa990e308" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35931 } } }
I20260811 19:04:26.351313 4424 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2624067b0904345bbe2a05aa990e308 [sys.catalog]: This master's current role is: LEADER
I20260811 19:04:26.351647 4423 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2624067b0904345bbe2a05aa990e308 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b2624067b0904345bbe2a05aa990e308. Latest consensus state: current_term: 1 leader_uuid: "b2624067b0904345bbe2a05aa990e308" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2624067b0904345bbe2a05aa990e308" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35931 } } }
I20260811 19:04:26.351735 4423 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2624067b0904345bbe2a05aa990e308 [sys.catalog]: This master's current role is: LEADER
I20260811 19:04:26.352659 4433 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:04:26.355897 4433 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:04:26.359339 4415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:26.349538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.104:43245 (local address 127.0.0.1:35931)
0811 19:04:26.349663 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:26.349666 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:26.349940 (+ 274us) server_negotiation.cc:407] Connection header received
0811 19:04:26.350121 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:26.350124 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:26.350176 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:26.350259 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:26.351112 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:26.351905 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:26.353021 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:26.353304 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:26.353419 (+ 115us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:26.354996 (+ 1577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:04:26.355000 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:04:26.355003 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:04:26.355032 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:04:26.356965 (+ 1933us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:26.357468 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:26.357472 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:26.357474 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:26.357561 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:26.357930 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:26.357933 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:26.357935 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:26.358178 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:04:26.358991 (+ 813us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:26.359097 (+ 106us) server_negotiation.cc:299] Negotiation successful
0811 19:04:26.359193 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 19:04:26.362756 4433 catalog_manager.cc:1347] Generated new cluster ID: 701b46b6a50c40969d8475be3fd20fa4
I20260811 19:04:26.362871 4433 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:04:26.377295 4433 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:04:26.378561 4433 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:04:26.386471 4433 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b2624067b0904345bbe2a05aa990e308: Generated new TSK 0
I20260811 19:04:26.387533 4433 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 19:04:26.401945 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 4332
W20260811 19:04:26.411754 4314 connection.cc:535] client connection to 127.0.0.1:35931 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35931): Connection reset by peer (error 104)
2026-08-11T19:04:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1548 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474792496492-13748-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:04:26 dist-test-slave-0kls krb5kdc[4447](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:04:26 dist-test-slave-0kls krb5kdc[4447](info): set up 2 sockets
Aug 11 19:04:26 dist-test-slave-0kls krb5kdc[4447](info): commencing operation
krb5kdc: starting...
W20260811 19:04:27.506271 13748 mini_kdc.cc:121] Time spent starting KDC: real 1.079s user 0.006s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474792496492-13748-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474792496492-13748-0/krb5kdc/test-user.keytab.
Aug 11 19:04:27 dist-test-slave-0kls krb5kdc[4447](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475067, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:04:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:04:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474792496492-13748-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474792496492-13748-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 19:04:27.648840 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474792496492-13748-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474792496492-13748-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474792496492-13748-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41195
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474792496492-13748-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41195
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:04:27.770042 4461 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:04:27.770411 4461 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:04:27.770499 4461 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:04:27.774745 4461 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:04:27.774874 4461 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:04:27.774916 4461 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:04:27.774956 4461 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:04:27.774993 4461 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:04:27.780910 4461 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474792496492-13748-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474792496492-13748-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41195
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474792496492-13748-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41195
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4461
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474792496492-13748-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:04:27.782269 4461 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:04:27.783447 4461 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:04:27.793143 4469 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:04:27.793162 4468 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:04:27.793345 4461 server_base.cc:1034] running on GCE node
W20260811 19:04:27.793195 4471 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:04:27.794348 4461 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:04:27.795820 4461 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:04:27.797012 4461 hybrid_clock.cc:648] HybridClock initialized: now 1786475067796990 us; error 66 us; skew 500 ppm
Aug 11 19:04:27 dist-test-slave-0kls krb5kdc[4447](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475067, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:04:27.800104 4461 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 19:04:27.801179 4461 webserver.cc:460] Webserver started at http://127.0.0.1:33913/ using document root <none> and password file <none>
I20260811 19:04:27.801864 4461 fs_manager.cc:359] Metadata directory not provided
I20260811 19:04:27.801937 4461 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:04:27.802193 4461 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:04:27.804392 4461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474792496492-13748-0/minicluster-data/master-0/data/instance:
uuid: "4d8cb8cf0ba64d1ba68c9d29fdeb25bf"
format_stamp: "Formatted at 2026-08-11 19:04:27 on dist-test-slave-0kls"
server_key: "4ed8d0b66fa8e79ccdfa161cadf02c17"
server_key_iv: "272427a2dde25d79e3591d39e5d9d769"
server_key_version: "encryptionkey@0"
I20260811 19:04:27.805008 4461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474792496492-13748-0/minicluster-data/master-0/wal/instance:
uuid: "4d8cb8cf0ba64d1ba68c9d29fdeb25bf"
format_stamp: "Formatted at 2026-08-11 19:04:27 on dist-test-slave-0kls"
server_key: "4ed8d0b66fa8e79ccdfa161cadf02c17"
server_key_iv: "272427a2dde25d79e3591d39e5d9d769"
server_key_version: "encryptionkey@0"
I20260811 19:04:27.809067 4461 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 19:04:27.811697 4478 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:04:27.812996 4461 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 19:04:27.813161 4461 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474792496492-13748-0/minicluster-data/master-0/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474792496492-13748-0/minicluster-data/master-0/wal
uuid: "4d8cb8cf0ba64d1ba68c9d29fdeb25bf"
format_stamp: "Formatted at 2026-08-11 19:04:27 on dist-test-slave-0kls"
server_key: "4ed8d0b66fa8e79ccdfa161cadf02c17"
server_key_iv: "272427a2dde25d79e3591d39e5d9d769"
server_key_version: "encryptionkey@0"
I20260811 19:04:27.813285 4461 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474792496492-13748-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474792496492-13748-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474792496492-13748-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:04:27.828622 4461 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:04:27.831736 4461 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:04:27.831960 4461 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:04:27.839639 4461 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41195
I20260811 19:04:27.839668 4543 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41195 every 8 connection(s)
I20260811 19:04:27.840664 4461 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
I20260811 19:04:27.843575 4544 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:04:27.847263 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 4461
I20260811 19:04:27.847419 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474792496492-13748-0/minicluster-data/master-0/wal/instance
I20260811 19:04:27.847728 13748 external_mini_cluster.cc:1428] Setting key 64f2fa9c4582cdb6e7d03c3687da063d
I20260811 19:04:27.850029 4544 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d8cb8cf0ba64d1ba68c9d29fdeb25bf: Bootstrap starting.
I20260811 19:04:27.852528 4544 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4d8cb8cf0ba64d1ba68c9d29fdeb25bf: Neither blocks nor log segments found. Creating new log.
I20260811 19:04:27.853374 4544 log.cc:824] T 00000000000000000000000000000000 P 4d8cb8cf0ba64d1ba68c9d29fdeb25bf: Log is configured to *not* fsync() on all Append() calls
I20260811 19:04:27.855662 4544 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d8cb8cf0ba64d1ba68c9d29fdeb25bf: No bootstrap required, opened a new log
I20260811 19:04:27.858235 4547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:27.849445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.104:49207 (local address 127.0.0.1:41195)
0811 19:04:27.850040 (+ 595us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:27.850059 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:27.850103 (+ 44us) server_negotiation.cc:407] Connection header received
0811 19:04:27.850880 (+ 777us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:27.850900 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:27.851296 (+ 396us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:27.851687 (+ 391us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:27.852177 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:27.853311 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:27.854411 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:27.854941 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:27.855160 (+ 219us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:27.856211 (+ 1051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:04:27.856217 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:04:27.856233 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:04:27.856667 (+ 434us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:04:27.856734 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:04:27.856760 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:04:27.856928 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:27.857145 (+ 217us) server_negotiation.cc:299] Negotiation successful
0811 19:04:27.857314 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":417,"thread_start_us":164,"threads_started":1}
I20260811 19:04:27.860023 4544 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4d8cb8cf0ba64d1ba68c9d29fdeb25bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d8cb8cf0ba64d1ba68c9d29fdeb25bf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41195 } }
I20260811 19:04:27.860301 4544 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4d8cb8cf0ba64d1ba68c9d29fdeb25bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:04:27.860419 4544 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4d8cb8cf0ba64d1ba68c9d29fdeb25bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d8cb8cf0ba64d1ba68c9d29fdeb25bf, State: Initialized, Role: FOLLOWER
I20260811 19:04:27.861016 4544 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4d8cb8cf0ba64d1ba68c9d29fdeb25bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d8cb8cf0ba64d1ba68c9d29fdeb25bf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41195 } }
I20260811 19:04:27.861212 4544 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4d8cb8cf0ba64d1ba68c9d29fdeb25bf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:04:27.861315 4544 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4d8cb8cf0ba64d1ba68c9d29fdeb25bf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:04:27.861436 4544 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4d8cb8cf0ba64d1ba68c9d29fdeb25bf [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:04:27.862020 13748 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 19:04:27.862690 4544 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4d8cb8cf0ba64d1ba68c9d29fdeb25bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d8cb8cf0ba64d1ba68c9d29fdeb25bf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41195 } }
I20260811 19:04:27.863087 4544 leader_election.cc:302] T 00000000000000000000000000000000 P 4d8cb8cf0ba64d1ba68c9d29fdeb25bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4d8cb8cf0ba64d1ba68c9d29fdeb25bf; no voters:
I20260811 19:04:27.863469 4544 leader_election.cc:288] T 00000000000000000000000000000000 P 4d8cb8cf0ba64d1ba68c9d29fdeb25bf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:04:27.863647 4552 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4d8cb8cf0ba64d1ba68c9d29fdeb25bf [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:04:27.863869 4552 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4d8cb8cf0ba64d1ba68c9d29fdeb25bf [term 1 LEADER]: Becoming Leader. State: Replica: 4d8cb8cf0ba64d1ba68c9d29fdeb25bf, State: Running, Role: LEADER
I20260811 19:04:27.864292 4552 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4d8cb8cf0ba64d1ba68c9d29fdeb25bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d8cb8cf0ba64d1ba68c9d29fdeb25bf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41195 } }
I20260811 19:04:27.865095 4544 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4d8cb8cf0ba64d1ba68c9d29fdeb25bf [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:04:27.866466 4555 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d8cb8cf0ba64d1ba68c9d29fdeb25bf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4d8cb8cf0ba64d1ba68c9d29fdeb25bf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d8cb8cf0ba64d1ba68c9d29fdeb25bf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41195 } } }
I20260811 19:04:27.866673 4555 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d8cb8cf0ba64d1ba68c9d29fdeb25bf [sys.catalog]: This master's current role is: LEADER
I20260811 19:04:27.866971 4555 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d8cb8cf0ba64d1ba68c9d29fdeb25bf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4d8cb8cf0ba64d1ba68c9d29fdeb25bf. Latest consensus state: current_term: 1 leader_uuid: "4d8cb8cf0ba64d1ba68c9d29fdeb25bf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d8cb8cf0ba64d1ba68c9d29fdeb25bf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41195 } } }
I20260811 19:04:27.867143 4555 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d8cb8cf0ba64d1ba68c9d29fdeb25bf [sys.catalog]: This master's current role is: LEADER
I20260811 19:04:27.869974 4547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:27.864598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.104:55411 (local address 127.0.0.1:41195)
0811 19:04:27.864844 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:27.864848 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:27.865169 (+ 321us) server_negotiation.cc:407] Connection header received
0811 19:04:27.865356 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:27.865360 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:27.865424 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:27.865531 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:27.865903 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:27.866674 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:27.867888 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:27.868229 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:27.868351 (+ 122us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:27.869316 (+ 965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:04:27.869319 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:04:27.869321 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:04:27.869334 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:04:27.869362 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:04:27.869375 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:04:27.869529 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:27.869734 (+ 205us) server_negotiation.cc:299] Negotiation successful
0811 19:04:27.869773 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165}
W20260811 19:04:27.871356 4571 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4d8cb8cf0ba64d1ba68c9d29fdeb25bf: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 19:04:27.871445 4571 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 19:04:27.871531 4566 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:04:27.872234 4566 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:04:27.878278 4566 catalog_manager.cc:1347] Generated new cluster ID: 8bbea61002c14eecb7c0028d51023d80
I20260811 19:04:27.878358 4566 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:04:27.893356 4566 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:04:27.894608 4566 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:04:27.899633 4566 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4d8cb8cf0ba64d1ba68c9d29fdeb25bf: Generated new TSK 0
I20260811 19:04:27.900345 4566 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 19:04:27.905061 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 4461
W20260811 19:04:27.913460 4445 connection.cc:535] client connection to 127.0.0.1:41195 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41195): Connection reset by peer (error 104)
2026-08-11T19:04:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1501 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474792496492-13748-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:04:27 dist-test-slave-0kls krb5kdc[4577](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:04:27 dist-test-slave-0kls krb5kdc[4577](info): set up 2 sockets
Aug 11 19:04:27 dist-test-slave-0kls krb5kdc[4577](info): commencing operation
krb5kdc: starting...
W20260811 19:04:29.022770 13748 mini_kdc.cc:121] Time spent starting KDC: real 1.094s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474792496492-13748-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474792496492-13748-0/krb5kdc/test-user.keytab.
Aug 11 19:04:29 dist-test-slave-0kls krb5kdc[4577](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475069, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:04:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:04:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474792496492-13748-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474792496492-13748-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 19:04:29.165112 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474792496492-13748-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474792496492-13748-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474792496492-13748-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32837
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474792496492-13748-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:32837
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:04:29.281412 4593 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:04:29.281769 4593 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:04:29.281874 4593 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:04:29.286446 4593 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:04:29.286574 4593 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:04:29.286616 4593 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:04:29.286655 4593 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:04:29.286691 4593 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:04:29.292508 4593 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474792496492-13748-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474792496492-13748-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:32837
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474792496492-13748-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32837
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4593
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474792496492-13748-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:04:29.293777 4593 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:04:29.294889 4593 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:04:29.303895 4600 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:04:29.303913 4603 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:04:29.303913 4601 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:04:29.304301 4593 server_base.cc:1034] running on GCE node
I20260811 19:04:29.304935 4593 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:04:29.306303 4593 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:04:29.307512 4593 hybrid_clock.cc:648] HybridClock initialized: now 1786475069307494 us; error 34 us; skew 500 ppm
Aug 11 19:04:29 dist-test-slave-0kls krb5kdc[4577](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475069, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:04:29.310724 4593 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 19:04:29.311821 4593 webserver.cc:460] Webserver started at http://127.0.0.1:43621/ using document root <none> and password file <none>
I20260811 19:04:29.312458 4593 fs_manager.cc:359] Metadata directory not provided
I20260811 19:04:29.312525 4593 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:04:29.312757 4593 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:04:29.315075 4593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474792496492-13748-0/minicluster-data/master-0/data/instance:
uuid: "258b591e1eb2499584a9ecf9f8583a8d"
format_stamp: "Formatted at 2026-08-11 19:04:29 on dist-test-slave-0kls"
server_key: "e495eb511ec795e69fdf0773b9d98c80"
server_key_iv: "84de8108d3e9f8d22b0b60704819ca84"
server_key_version: "encryptionkey@0"
I20260811 19:04:29.315706 4593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474792496492-13748-0/minicluster-data/master-0/wal/instance:
uuid: "258b591e1eb2499584a9ecf9f8583a8d"
format_stamp: "Formatted at 2026-08-11 19:04:29 on dist-test-slave-0kls"
server_key: "e495eb511ec795e69fdf0773b9d98c80"
server_key_iv: "84de8108d3e9f8d22b0b60704819ca84"
server_key_version: "encryptionkey@0"
I20260811 19:04:29.320164 4593 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 19:04:29.323288 4613 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:04:29.325001 4593 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 19:04:29.325207 4593 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474792496492-13748-0/minicluster-data/master-0/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474792496492-13748-0/minicluster-data/master-0/wal
uuid: "258b591e1eb2499584a9ecf9f8583a8d"
format_stamp: "Formatted at 2026-08-11 19:04:29 on dist-test-slave-0kls"
server_key: "e495eb511ec795e69fdf0773b9d98c80"
server_key_iv: "84de8108d3e9f8d22b0b60704819ca84"
server_key_version: "encryptionkey@0"
I20260811 19:04:29.325363 4593 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474792496492-13748-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474792496492-13748-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474792496492-13748-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:04:29.330775 4593 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:04:29.335062 4593 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:04:29.335436 4593 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:04:29.343961 4593 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:32837
I20260811 19:04:29.343973 4673 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:32837 every 8 connection(s)
I20260811 19:04:29.345129 4593 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
I20260811 19:04:29.348547 4674 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:04:29.352965 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 4593
I20260811 19:04:29.353087 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474792496492-13748-0/minicluster-data/master-0/wal/instance
I20260811 19:04:29.353417 13748 external_mini_cluster.cc:1428] Setting key cebfc17b34edbfccb5f52d5993f3a6aa
I20260811 19:04:29.355405 4674 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 258b591e1eb2499584a9ecf9f8583a8d: Bootstrap starting.
I20260811 19:04:29.358096 4674 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 258b591e1eb2499584a9ecf9f8583a8d: Neither blocks nor log segments found. Creating new log.
I20260811 19:04:29.359180 4674 log.cc:824] T 00000000000000000000000000000000 P 258b591e1eb2499584a9ecf9f8583a8d: Log is configured to *not* fsync() on all Append() calls
I20260811 19:04:29.359961 4677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:29.355160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.104:40895 (local address 127.0.0.1:32837)
0811 19:04:29.355749 (+ 589us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:29.355766 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:29.355808 (+ 42us) server_negotiation.cc:407] Connection header received
0811 19:04:29.356585 (+ 777us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:29.356604 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:29.356941 (+ 337us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:29.357345 (+ 404us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:29.358332 (+ 987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:04:29.358341 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:04:29.358362 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:04:29.358377 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:04:29.358465 (+ 88us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:04:29.358496 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:04:29.358633 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:29.358761 (+ 128us) server_negotiation.cc:299] Negotiation successful
0811 19:04:29.358945 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":417,"thread_start_us":162,"threads_started":1}
I20260811 19:04:29.361610 4674 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 258b591e1eb2499584a9ecf9f8583a8d: No bootstrap required, opened a new log
I20260811 19:04:29.363144 13748 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 19:04:29.366238 4674 raft_consensus.cc:348] T 00000000000000000000000000000000 P 258b591e1eb2499584a9ecf9f8583a8d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "258b591e1eb2499584a9ecf9f8583a8d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32837 } }
I20260811 19:04:29.366549 4674 raft_consensus.cc:374] T 00000000000000000000000000000000 P 258b591e1eb2499584a9ecf9f8583a8d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:04:29.366616 4674 raft_consensus.cc:729] T 00000000000000000000000000000000 P 258b591e1eb2499584a9ecf9f8583a8d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 258b591e1eb2499584a9ecf9f8583a8d, State: Initialized, Role: FOLLOWER
I20260811 19:04:29.367164 4674 consensus_queue.cc:260] T 00000000000000000000000000000000 P 258b591e1eb2499584a9ecf9f8583a8d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "258b591e1eb2499584a9ecf9f8583a8d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32837 } }
I20260811 19:04:29.367324 4677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:29.365420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.104:44181 (local address 127.0.0.1:32837)
0811 19:04:29.365591 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:29.365596 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:29.365741 (+ 145us) server_negotiation.cc:407] Connection header received
0811 19:04:29.365930 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:29.365933 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:29.365981 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:29.366102 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:29.366887 (+ 785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:04:29.366890 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:04:29.366893 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:04:29.366906 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:04:29.366942 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:04:29.366953 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:04:29.367069 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:29.367143 (+ 74us) server_negotiation.cc:299] Negotiation successful
0811 19:04:29.367191 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260811 19:04:29.367358 4674 raft_consensus.cc:388] T 00000000000000000000000000000000 P 258b591e1eb2499584a9ecf9f8583a8d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:04:29.367442 4674 raft_consensus.cc:482] T 00000000000000000000000000000000 P 258b591e1eb2499584a9ecf9f8583a8d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:04:29.367569 4674 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 258b591e1eb2499584a9ecf9f8583a8d [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:04:29.368754 4674 raft_consensus.cc:504] T 00000000000000000000000000000000 P 258b591e1eb2499584a9ecf9f8583a8d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "258b591e1eb2499584a9ecf9f8583a8d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32837 } }
I20260811 19:04:29.369107 4674 leader_election.cc:302] T 00000000000000000000000000000000 P 258b591e1eb2499584a9ecf9f8583a8d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 258b591e1eb2499584a9ecf9f8583a8d; no voters:
I20260811 19:04:29.369416 4674 leader_election.cc:288] T 00000000000000000000000000000000 P 258b591e1eb2499584a9ecf9f8583a8d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:04:29.369550 4686 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 258b591e1eb2499584a9ecf9f8583a8d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:04:29.369867 4686 raft_consensus.cc:686] T 00000000000000000000000000000000 P 258b591e1eb2499584a9ecf9f8583a8d [term 1 LEADER]: Becoming Leader. State: Replica: 258b591e1eb2499584a9ecf9f8583a8d, State: Running, Role: LEADER
I20260811 19:04:29.370342 4686 consensus_queue.cc:237] T 00000000000000000000000000000000 P 258b591e1eb2499584a9ecf9f8583a8d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "258b591e1eb2499584a9ecf9f8583a8d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32837 } }
I20260811 19:04:29.370925 4674 sys_catalog.cc:564] T 00000000000000000000000000000000 P 258b591e1eb2499584a9ecf9f8583a8d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:04:29.372133 4688 sys_catalog.cc:455] T 00000000000000000000000000000000 P 258b591e1eb2499584a9ecf9f8583a8d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 258b591e1eb2499584a9ecf9f8583a8d. Latest consensus state: current_term: 1 leader_uuid: "258b591e1eb2499584a9ecf9f8583a8d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "258b591e1eb2499584a9ecf9f8583a8d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32837 } } }
I20260811 19:04:29.372279 4688 sys_catalog.cc:458] T 00000000000000000000000000000000 P 258b591e1eb2499584a9ecf9f8583a8d [sys.catalog]: This master's current role is: LEADER
I20260811 19:04:29.372776 4695 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:04:29.372773 4687 sys_catalog.cc:455] T 00000000000000000000000000000000 P 258b591e1eb2499584a9ecf9f8583a8d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "258b591e1eb2499584a9ecf9f8583a8d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "258b591e1eb2499584a9ecf9f8583a8d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32837 } } }
I20260811 19:04:29.372928 4687 sys_catalog.cc:458] T 00000000000000000000000000000000 P 258b591e1eb2499584a9ecf9f8583a8d [sys.catalog]: This master's current role is: LEADER
I20260811 19:04:29.375857 4695 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:04:29.381757 4695 catalog_manager.cc:1347] Generated new cluster ID: 00220b58f9d341e98025af89e5c67415
I20260811 19:04:29.381901 4695 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:04:29.393807 4695 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:04:29.394237 4695 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:04:29.401222 4695 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 258b591e1eb2499584a9ecf9f8583a8d: Generated new TSK 0
I20260811 19:04:29.402189 4695 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 19:04:29.407032 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 4593
2026-08-11T19:04:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1501 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474792496492-13748-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:04:29 dist-test-slave-0kls krb5kdc[4707](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:04:29 dist-test-slave-0kls krb5kdc[4707](info): set up 2 sockets
Aug 11 19:04:29 dist-test-slave-0kls krb5kdc[4707](info): commencing operation
krb5kdc: starting...
W20260811 19:04:30.511785 13748 mini_kdc.cc:121] Time spent starting KDC: real 1.082s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474792496492-13748-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474792496492-13748-0/krb5kdc/test-user.keytab.
Aug 11 19:04:30 dist-test-slave-0kls krb5kdc[4707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475070, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:04:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:04:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474792496492-13748-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474792496492-13748-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 19:04:30.654006 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474792496492-13748-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474792496492-13748-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474792496492-13748-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38959
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474792496492-13748-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38959
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:04:30.777578 4722 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:04:30.778023 4722 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:04:30.778152 4722 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:04:30.782579 4722 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:04:30.782701 4722 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:04:30.782743 4722 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:04:30.782783 4722 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:04:30.782826 4722 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:04:30.788722 4722 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474792496492-13748-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474792496492-13748-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38959
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474792496492-13748-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38959
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4722
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474792496492-13748-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:04:30.790140 4722 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:04:30.791352 4722 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:04:30.801047 4731 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:04:30.801244 4722 server_base.cc:1034] running on GCE node
W20260811 19:04:30.801072 4729 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:04:30.801072 4727 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:04:30.802204 4722 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:04:30.803584 4722 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:04:30.804792 4722 hybrid_clock.cc:648] HybridClock initialized: now 1786475070804775 us; error 34 us; skew 500 ppm
Aug 11 19:04:30 dist-test-slave-0kls krb5kdc[4707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475070, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:04:30.807878 4722 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 19:04:30.808889 4722 webserver.cc:460] Webserver started at http://127.0.0.1:33311/ using document root <none> and password file <none>
I20260811 19:04:30.809480 4722 fs_manager.cc:359] Metadata directory not provided
I20260811 19:04:30.809551 4722 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:04:30.809751 4722 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:04:30.812183 4722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474792496492-13748-0/minicluster-data/master-0/data/instance:
uuid: "9fcad4b802e84d4e9066c9b5da13d2e6"
format_stamp: "Formatted at 2026-08-11 19:04:30 on dist-test-slave-0kls"
server_key: "1b48138980bbc18ebd43dff300e94ba0"
server_key_iv: "6e3b10bc797f844875d8eec134010737"
server_key_version: "encryptionkey@0"
I20260811 19:04:30.812815 4722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474792496492-13748-0/minicluster-data/master-0/wal/instance:
uuid: "9fcad4b802e84d4e9066c9b5da13d2e6"
format_stamp: "Formatted at 2026-08-11 19:04:30 on dist-test-slave-0kls"
server_key: "1b48138980bbc18ebd43dff300e94ba0"
server_key_iv: "6e3b10bc797f844875d8eec134010737"
server_key_version: "encryptionkey@0"
I20260811 19:04:30.817196 4722 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260811 19:04:30.820088 4742 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:04:30.821585 4722 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 19:04:30.821775 4722 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474792496492-13748-0/minicluster-data/master-0/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474792496492-13748-0/minicluster-data/master-0/wal
uuid: "9fcad4b802e84d4e9066c9b5da13d2e6"
format_stamp: "Formatted at 2026-08-11 19:04:30 on dist-test-slave-0kls"
server_key: "1b48138980bbc18ebd43dff300e94ba0"
server_key_iv: "6e3b10bc797f844875d8eec134010737"
server_key_version: "encryptionkey@0"
I20260811 19:04:30.821996 4722 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474792496492-13748-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474792496492-13748-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474792496492-13748-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:04:30.848970 4722 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:04:30.852772 4722 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:04:30.853037 4722 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:04:30.861073 4722 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38959
I20260811 19:04:30.861115 4801 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38959 every 8 connection(s)
I20260811 19:04:30.862269 4722 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
I20260811 19:04:30.862687 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 4722
I20260811 19:04:30.862852 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474792496492-13748-0/minicluster-data/master-0/wal/instance
I20260811 19:04:30.863175 13748 external_mini_cluster.cc:1428] Setting key 316239a3aa91eba49769f5d92ac3618a
I20260811 19:04:30.865736 4802 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:04:30.871670 4802 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9fcad4b802e84d4e9066c9b5da13d2e6: Bootstrap starting.
Aug 11 19:04:30 dist-test-slave-0kls krb5kdc[4707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475070, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 19:04:30.874390 4802 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9fcad4b802e84d4e9066c9b5da13d2e6: Neither blocks nor log segments found. Creating new log.
I20260811 19:04:30.875289 4802 log.cc:824] T 00000000000000000000000000000000 P 9fcad4b802e84d4e9066c9b5da13d2e6: Log is configured to *not* fsync() on all Append() calls
I20260811 19:04:30.877355 4802 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9fcad4b802e84d4e9066c9b5da13d2e6: No bootstrap required, opened a new log
I20260811 19:04:30.878489 4805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:30.864742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.104:35219 (local address 127.0.0.1:38959)
0811 19:04:30.865232 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:30.865249 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:30.865286 (+ 37us) server_negotiation.cc:407] Connection header received
0811 19:04:30.866065 (+ 779us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:30.866090 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:30.866450 (+ 360us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:30.866824 (+ 374us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:30.867670 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:30.868671 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:30.869615 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:30.870164 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:30.870359 (+ 195us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:30.873028 (+ 2669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:04:30.873033 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:04:30.873048 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:04:30.873078 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:04:30.875187 (+ 2109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:30.875775 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:30.875781 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:30.875788 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:30.875919 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:30.876297 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:30.876300 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:30.876302 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:30.876740 (+ 438us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:04:30.876972 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:30.877353 (+ 381us) server_negotiation.cc:299] Negotiation successful
0811 19:04:30.877624 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":157,"threads_started":1}
I20260811 19:04:30.881301 4802 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9fcad4b802e84d4e9066c9b5da13d2e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fcad4b802e84d4e9066c9b5da13d2e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38959 } }
I20260811 19:04:30.881578 4802 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9fcad4b802e84d4e9066c9b5da13d2e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:04:30.881651 4802 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9fcad4b802e84d4e9066c9b5da13d2e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9fcad4b802e84d4e9066c9b5da13d2e6, State: Initialized, Role: FOLLOWER
I20260811 19:04:30.882194 4802 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9fcad4b802e84d4e9066c9b5da13d2e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fcad4b802e84d4e9066c9b5da13d2e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38959 } }
I20260811 19:04:30.882400 4802 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9fcad4b802e84d4e9066c9b5da13d2e6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:04:30.882478 4802 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9fcad4b802e84d4e9066c9b5da13d2e6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:04:30.882534 13748 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 19:04:30.882608 4802 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9fcad4b802e84d4e9066c9b5da13d2e6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:04:30.883670 4802 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9fcad4b802e84d4e9066c9b5da13d2e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fcad4b802e84d4e9066c9b5da13d2e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38959 } }
I20260811 19:04:30.884115 4802 leader_election.cc:302] T 00000000000000000000000000000000 P 9fcad4b802e84d4e9066c9b5da13d2e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9fcad4b802e84d4e9066c9b5da13d2e6; no voters:
I20260811 19:04:30.884552 4802 leader_election.cc:288] T 00000000000000000000000000000000 P 9fcad4b802e84d4e9066c9b5da13d2e6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:04:30.884682 4813 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9fcad4b802e84d4e9066c9b5da13d2e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:04:30.884956 4813 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9fcad4b802e84d4e9066c9b5da13d2e6 [term 1 LEADER]: Becoming Leader. State: Replica: 9fcad4b802e84d4e9066c9b5da13d2e6, State: Running, Role: LEADER
I20260811 19:04:30.885377 4813 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9fcad4b802e84d4e9066c9b5da13d2e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fcad4b802e84d4e9066c9b5da13d2e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38959 } }
I20260811 19:04:30.886073 4802 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9fcad4b802e84d4e9066c9b5da13d2e6 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:04:30.887399 4814 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9fcad4b802e84d4e9066c9b5da13d2e6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9fcad4b802e84d4e9066c9b5da13d2e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fcad4b802e84d4e9066c9b5da13d2e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38959 } } }
I20260811 19:04:30.887545 4814 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9fcad4b802e84d4e9066c9b5da13d2e6 [sys.catalog]: This master's current role is: LEADER
I20260811 19:04:30.887755 4815 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9fcad4b802e84d4e9066c9b5da13d2e6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9fcad4b802e84d4e9066c9b5da13d2e6. Latest consensus state: current_term: 1 leader_uuid: "9fcad4b802e84d4e9066c9b5da13d2e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fcad4b802e84d4e9066c9b5da13d2e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38959 } } }
I20260811 19:04:30.887950 4815 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9fcad4b802e84d4e9066c9b5da13d2e6 [sys.catalog]: This master's current role is: LEADER
I20260811 19:04:30.889016 4829 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:04:30.892745 4829 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:04:30.895581 4805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:30.884805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.104:45197 (local address 127.0.0.1:38959)
0811 19:04:30.884948 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:30.884953 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:30.885620 (+ 667us) server_negotiation.cc:407] Connection header received
0811 19:04:30.885803 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:30.885807 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:30.885866 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:30.885959 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:30.888115 (+ 2156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:30.888907 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:30.889842 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:30.890173 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:30.890297 (+ 124us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:30.891881 (+ 1584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:04:30.891885 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:04:30.891888 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:04:30.891927 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:04:30.893429 (+ 1502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:30.894049 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:30.894053 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:30.894055 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:30.894187 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:30.894575 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:30.894579 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:30.894580 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:30.894792 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:04:30.894956 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:30.895281 (+ 325us) server_negotiation.cc:299] Negotiation successful
0811 19:04:30.895404 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 19:04:30.899914 4829 catalog_manager.cc:1347] Generated new cluster ID: f20bb3fc39db4569aece46b347354964
I20260811 19:04:30.900020 4829 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:04:30.909809 4829 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:04:30.910991 4829 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:04:30.917654 4829 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9fcad4b802e84d4e9066c9b5da13d2e6: Generated new TSK 0
I20260811 19:04:30.918609 4829 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 19:04:30.923624 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 4722
W20260811 19:04:30.934022 4705 connection.cc:535] client connection to 127.0.0.1:38959 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38959): Connection reset by peer (error 104)
2026-08-11T19:04:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1519 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474792496492-13748-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:04:30 dist-test-slave-0kls krb5kdc[4837](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:04:30 dist-test-slave-0kls krb5kdc[4837](info): set up 2 sockets
Aug 11 19:04:30 dist-test-slave-0kls krb5kdc[4837](info): commencing operation
krb5kdc: starting...
W20260811 19:04:32.029675 13748 mini_kdc.cc:121] Time spent starting KDC: real 1.081s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474792496492-13748-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474792496492-13748-0/krb5kdc/test-user.keytab.
Aug 11 19:04:32 dist-test-slave-0kls krb5kdc[4837](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475072, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:04:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:04:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474792496492-13748-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474792496492-13748-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 19:04:32.167008 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474792496492-13748-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474792496492-13748-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474792496492-13748-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41363
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474792496492-13748-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41363
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:04:32.289911 4851 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:04:32.290227 4851 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:04:32.290277 4851 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:04:32.294567 4851 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:04:32.294637 4851 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:04:32.294657 4851 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:04:32.294700 4851 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:04:32.294715 4851 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:04:32.300549 4851 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474792496492-13748-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474792496492-13748-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41363
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474792496492-13748-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41363
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4851
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474792496492-13748-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:04:32.301700 4851 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:04:32.302853 4851 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:04:32.312361 4856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:04:32.312417 4860 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:04:32.312417 4858 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:04:32.312770 4851 server_base.cc:1034] running on GCE node
I20260811 19:04:32.313612 4851 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:04:32.315081 4851 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:04:32.316280 4851 hybrid_clock.cc:648] HybridClock initialized: now 1786475072316247 us; error 62 us; skew 500 ppm
Aug 11 19:04:32 dist-test-slave-0kls krb5kdc[4837](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475072, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:04:32.324152 4851 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 19:04:32.325433 4851 webserver.cc:460] Webserver started at http://127.0.0.1:38813/ using document root <none> and password file <none>
I20260811 19:04:32.326210 4851 fs_manager.cc:359] Metadata directory not provided
I20260811 19:04:32.326324 4851 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:04:32.326612 4851 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:04:32.329010 4851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474792496492-13748-0/minicluster-data/master-0/data/instance:
uuid: "905c4adab09b4bddbc7bb0b35443c0fc"
format_stamp: "Formatted at 2026-08-11 19:04:32 on dist-test-slave-0kls"
server_key: "c885edacd43375b6e92591ce117d1a55"
server_key_iv: "5d01763135f5cced4cd0bbfc5a64389e"
server_key_version: "encryptionkey@0"
I20260811 19:04:32.329710 4851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474792496492-13748-0/minicluster-data/master-0/wal/instance:
uuid: "905c4adab09b4bddbc7bb0b35443c0fc"
format_stamp: "Formatted at 2026-08-11 19:04:32 on dist-test-slave-0kls"
server_key: "c885edacd43375b6e92591ce117d1a55"
server_key_iv: "5d01763135f5cced4cd0bbfc5a64389e"
server_key_version: "encryptionkey@0"
I20260811 19:04:32.334169 4851 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 19:04:32.337075 4867 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:04:32.338706 4851 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 19:04:32.338897 4851 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474792496492-13748-0/minicluster-data/master-0/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474792496492-13748-0/minicluster-data/master-0/wal
uuid: "905c4adab09b4bddbc7bb0b35443c0fc"
format_stamp: "Formatted at 2026-08-11 19:04:32 on dist-test-slave-0kls"
server_key: "c885edacd43375b6e92591ce117d1a55"
server_key_iv: "5d01763135f5cced4cd0bbfc5a64389e"
server_key_version: "encryptionkey@0"
I20260811 19:04:32.339047 4851 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474792496492-13748-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474792496492-13748-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474792496492-13748-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:04:32.360924 4851 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:04:32.364782 4851 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:04:32.365018 4851 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:04:32.373239 4851 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41363
I20260811 19:04:32.373255 4925 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41363 every 8 connection(s)
I20260811 19:04:32.374393 4851 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
I20260811 19:04:32.375573 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 4851
I20260811 19:04:32.375728 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474792496492-13748-0/minicluster-data/master-0/wal/instance
I20260811 19:04:32.376031 13748 external_mini_cluster.cc:1428] Setting key e2afc786fe195f9cc30fbbe43b57307f
I20260811 19:04:32.377899 4926 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:04:32.384799 4926 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 905c4adab09b4bddbc7bb0b35443c0fc: Bootstrap starting.
I20260811 19:04:32.386399 4929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:32.377579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.104:46359 (local address 127.0.0.1:41363)
0811 19:04:32.378221 (+ 642us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:32.378249 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:32.378291 (+ 42us) server_negotiation.cc:407] Connection header received
0811 19:04:32.378967 (+ 676us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:32.378986 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:32.379373 (+ 387us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:32.379779 (+ 406us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:32.380252 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:32.381315 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:32.382396 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:32.382943 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:32.383153 (+ 210us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:32.384215 (+ 1062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:04:32.384220 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:04:32.384236 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:04:32.384666 (+ 430us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:04:32.384731 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:04:32.384756 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:04:32.384910 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:32.385155 (+ 245us) server_negotiation.cc:299] Negotiation successful
0811 19:04:32.385334 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":407,"thread_start_us":214,"threads_started":1}
I20260811 19:04:32.387981 4926 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 905c4adab09b4bddbc7bb0b35443c0fc: Neither blocks nor log segments found. Creating new log.
I20260811 19:04:32.388963 4926 log.cc:824] T 00000000000000000000000000000000 P 905c4adab09b4bddbc7bb0b35443c0fc: Log is configured to *not* fsync() on all Append() calls
I20260811 19:04:32.390133 13748 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 19:04:32.392061 4926 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 905c4adab09b4bddbc7bb0b35443c0fc: No bootstrap required, opened a new log
I20260811 19:04:32.397320 4929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:32.392705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.104:45847 (local address 127.0.0.1:41363)
0811 19:04:32.392856 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:32.392861 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:32.393059 (+ 198us) server_negotiation.cc:407] Connection header received
0811 19:04:32.393194 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:32.393196 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:32.393244 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:32.393313 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:32.393638 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:32.394220 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:32.395443 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:32.395655 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:32.395732 (+ 77us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:32.396684 (+ 952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:04:32.396687 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:04:32.396689 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:04:32.396703 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:04:32.396729 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:04:32.396769 (+ 40us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:04:32.396851 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:32.397033 (+ 182us) server_negotiation.cc:299] Negotiation successful
0811 19:04:32.397101 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 19:04:32.397535 4926 raft_consensus.cc:348] T 00000000000000000000000000000000 P 905c4adab09b4bddbc7bb0b35443c0fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "905c4adab09b4bddbc7bb0b35443c0fc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41363 } }
I20260811 19:04:32.397938 4926 raft_consensus.cc:374] T 00000000000000000000000000000000 P 905c4adab09b4bddbc7bb0b35443c0fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:04:32.398041 4926 raft_consensus.cc:729] T 00000000000000000000000000000000 P 905c4adab09b4bddbc7bb0b35443c0fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 905c4adab09b4bddbc7bb0b35443c0fc, State: Initialized, Role: FOLLOWER
I20260811 19:04:32.398581 4926 consensus_queue.cc:260] T 00000000000000000000000000000000 P 905c4adab09b4bddbc7bb0b35443c0fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "905c4adab09b4bddbc7bb0b35443c0fc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41363 } }
I20260811 19:04:32.398770 4926 raft_consensus.cc:388] T 00000000000000000000000000000000 P 905c4adab09b4bddbc7bb0b35443c0fc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:04:32.398867 4926 raft_consensus.cc:482] T 00000000000000000000000000000000 P 905c4adab09b4bddbc7bb0b35443c0fc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:04:32.399013 4926 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 905c4adab09b4bddbc7bb0b35443c0fc [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:04:32.400158 4926 raft_consensus.cc:504] T 00000000000000000000000000000000 P 905c4adab09b4bddbc7bb0b35443c0fc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "905c4adab09b4bddbc7bb0b35443c0fc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41363 } }
I20260811 19:04:32.400641 4926 leader_election.cc:302] T 00000000000000000000000000000000 P 905c4adab09b4bddbc7bb0b35443c0fc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 905c4adab09b4bddbc7bb0b35443c0fc; no voters:
I20260811 19:04:32.401064 4926 leader_election.cc:288] T 00000000000000000000000000000000 P 905c4adab09b4bddbc7bb0b35443c0fc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:04:32.401204 4936 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 905c4adab09b4bddbc7bb0b35443c0fc [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:04:32.401458 4936 raft_consensus.cc:686] T 00000000000000000000000000000000 P 905c4adab09b4bddbc7bb0b35443c0fc [term 1 LEADER]: Becoming Leader. State: Replica: 905c4adab09b4bddbc7bb0b35443c0fc, State: Running, Role: LEADER
I20260811 19:04:32.401937 4936 consensus_queue.cc:237] T 00000000000000000000000000000000 P 905c4adab09b4bddbc7bb0b35443c0fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "905c4adab09b4bddbc7bb0b35443c0fc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41363 } }
I20260811 19:04:32.402647 4926 sys_catalog.cc:564] T 00000000000000000000000000000000 P 905c4adab09b4bddbc7bb0b35443c0fc [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:04:32.403959 4937 sys_catalog.cc:455] T 00000000000000000000000000000000 P 905c4adab09b4bddbc7bb0b35443c0fc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "905c4adab09b4bddbc7bb0b35443c0fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "905c4adab09b4bddbc7bb0b35443c0fc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41363 } } }
I20260811 19:04:32.404106 4937 sys_catalog.cc:458] T 00000000000000000000000000000000 P 905c4adab09b4bddbc7bb0b35443c0fc [sys.catalog]: This master's current role is: LEADER
I20260811 19:04:32.404937 4950 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:04:32.405391 4938 sys_catalog.cc:455] T 00000000000000000000000000000000 P 905c4adab09b4bddbc7bb0b35443c0fc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 905c4adab09b4bddbc7bb0b35443c0fc. Latest consensus state: current_term: 1 leader_uuid: "905c4adab09b4bddbc7bb0b35443c0fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "905c4adab09b4bddbc7bb0b35443c0fc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41363 } } }
I20260811 19:04:32.405483 4938 sys_catalog.cc:458] T 00000000000000000000000000000000 P 905c4adab09b4bddbc7bb0b35443c0fc [sys.catalog]: This master's current role is: LEADER
I20260811 19:04:32.408317 4950 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:04:32.414759 4950 catalog_manager.cc:1347] Generated new cluster ID: 7da21de83d704e44a5b83b2f9348d0ff
I20260811 19:04:32.414860 4950 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:04:32.434830 4950 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:04:32.435971 4950 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:04:32.442291 4950 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 905c4adab09b4bddbc7bb0b35443c0fc: Generated new TSK 0
I20260811 19:04:32.443323 4950 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 19:04:32.453958 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 4851
W20260811 19:04:32.463892 4835 connection.cc:535] client connection to 127.0.0.1:41363 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41363): Connection reset by peer (error 104)
2026-08-11T19:04:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1529 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474792496492-13748-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:04:32 dist-test-slave-0kls krb5kdc[4963](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:04:32 dist-test-slave-0kls krb5kdc[4963](info): set up 2 sockets
Aug 11 19:04:32 dist-test-slave-0kls krb5kdc[4963](info): commencing operation
krb5kdc: starting...
W20260811 19:04:33.554935 13748 mini_kdc.cc:121] Time spent starting KDC: real 1.076s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474792496492-13748-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474792496492-13748-0/krb5kdc/test-user.keytab.
Aug 11 19:04:33 dist-test-slave-0kls krb5kdc[4963](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475073, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:04:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:04:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474792496492-13748-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474792496492-13748-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 19:04:33.697377 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474792496492-13748-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474792496492-13748-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474792496492-13748-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45947
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474792496492-13748-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45947
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:04:33.820077 4977 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:04:33.820422 4977 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:04:33.820506 4977 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:04:33.824647 4977 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:04:33.824750 4977 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:04:33.824787 4977 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:04:33.824851 4977 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:04:33.824909 4977 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:04:33.830595 4977 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474792496492-13748-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474792496492-13748-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45947
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474792496492-13748-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45947
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4977
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474792496492-13748-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:04:33.831808 4977 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:04:33.832890 4977 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 19:04:33.841843 4977 server_base.cc:1034] running on GCE node
W20260811 19:04:33.841732 4986 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:04:33.841738 4983 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:04:33.841725 4984 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:04:33.842739 4977 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:04:33.843930 4977 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:04:33.845129 4977 hybrid_clock.cc:648] HybridClock initialized: now 1786475073845108 us; error 35 us; skew 500 ppm
Aug 11 19:04:33 dist-test-slave-0kls krb5kdc[4963](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475073, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:04:33.848037 4977 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 19:04:33.849041 4977 webserver.cc:460] Webserver started at http://127.0.0.1:41475/ using document root <none> and password file <none>
I20260811 19:04:33.849651 4977 fs_manager.cc:359] Metadata directory not provided
I20260811 19:04:33.849750 4977 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:04:33.850026 4977 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:04:33.852203 4977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474792496492-13748-0/minicluster-data/master-0/data/instance:
uuid: "d68894b0170b4667bb9bb231b2a0c406"
format_stamp: "Formatted at 2026-08-11 19:04:33 on dist-test-slave-0kls"
server_key: "6f4057365f63d91a93ae2e4167edcec2"
server_key_iv: "cc74fb6b17b849be3bde74dfa0ee3b2c"
server_key_version: "encryptionkey@0"
I20260811 19:04:33.852831 4977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474792496492-13748-0/minicluster-data/master-0/wal/instance:
uuid: "d68894b0170b4667bb9bb231b2a0c406"
format_stamp: "Formatted at 2026-08-11 19:04:33 on dist-test-slave-0kls"
server_key: "6f4057365f63d91a93ae2e4167edcec2"
server_key_iv: "cc74fb6b17b849be3bde74dfa0ee3b2c"
server_key_version: "encryptionkey@0"
I20260811 19:04:33.857024 4977 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260811 19:04:33.859882 4994 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:04:33.861251 4977 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 19:04:33.861413 4977 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474792496492-13748-0/minicluster-data/master-0/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474792496492-13748-0/minicluster-data/master-0/wal
uuid: "d68894b0170b4667bb9bb231b2a0c406"
format_stamp: "Formatted at 2026-08-11 19:04:33 on dist-test-slave-0kls"
server_key: "6f4057365f63d91a93ae2e4167edcec2"
server_key_iv: "cc74fb6b17b849be3bde74dfa0ee3b2c"
server_key_version: "encryptionkey@0"
I20260811 19:04:33.861546 4977 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474792496492-13748-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474792496492-13748-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474792496492-13748-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:04:33.866488 4977 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:04:33.869433 4977 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:04:33.869654 4977 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:04:33.877189 4977 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45947
I20260811 19:04:33.877218 5053 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45947 every 8 connection(s)
I20260811 19:04:33.878261 4977 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
I20260811 19:04:33.881223 5054 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:04:33.886593 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 4977
I20260811 19:04:33.886755 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474792496492-13748-0/minicluster-data/master-0/wal/instance
I20260811 19:04:33.887053 13748 external_mini_cluster.cc:1428] Setting key 456a7d1c7549f330b984046b4dc7e4e8
I20260811 19:04:33.887485 5054 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d68894b0170b4667bb9bb231b2a0c406: Bootstrap starting.
I20260811 19:04:33.890275 5054 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d68894b0170b4667bb9bb231b2a0c406: Neither blocks nor log segments found. Creating new log.
I20260811 19:04:33.891223 5054 log.cc:824] T 00000000000000000000000000000000 P d68894b0170b4667bb9bb231b2a0c406: Log is configured to *not* fsync() on all Append() calls
I20260811 19:04:33.892711 5057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:33.888534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.104:48459 (local address 127.0.0.1:45947)
0811 19:04:33.889011 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:33.889027 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:33.889064 (+ 37us) server_negotiation.cc:407] Connection header received
0811 19:04:33.889715 (+ 651us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:33.889730 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:33.890085 (+ 355us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:33.890430 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:33.891219 (+ 789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:04:33.891224 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:04:33.891237 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:04:33.891248 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:04:33.891302 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:04:33.891321 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:04:33.891455 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:33.891621 (+ 166us) server_negotiation.cc:299] Negotiation successful
0811 19:04:33.891810 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":140,"threads_started":1}
I20260811 19:04:33.893596 5054 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d68894b0170b4667bb9bb231b2a0c406: No bootstrap required, opened a new log
I20260811 19:04:33.895691 13748 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 19:04:33.897768 5054 raft_consensus.cc:348] T 00000000000000000000000000000000 P d68894b0170b4667bb9bb231b2a0c406 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d68894b0170b4667bb9bb231b2a0c406" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45947 } }
I20260811 19:04:33.898160 5054 raft_consensus.cc:374] T 00000000000000000000000000000000 P d68894b0170b4667bb9bb231b2a0c406 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:04:33.898240 5054 raft_consensus.cc:729] T 00000000000000000000000000000000 P d68894b0170b4667bb9bb231b2a0c406 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d68894b0170b4667bb9bb231b2a0c406, State: Initialized, Role: FOLLOWER
I20260811 19:04:33.898808 5054 consensus_queue.cc:260] T 00000000000000000000000000000000 P d68894b0170b4667bb9bb231b2a0c406 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d68894b0170b4667bb9bb231b2a0c406" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45947 } }
I20260811 19:04:33.899019 5054 raft_consensus.cc:388] T 00000000000000000000000000000000 P d68894b0170b4667bb9bb231b2a0c406 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:04:33.899094 5054 raft_consensus.cc:482] T 00000000000000000000000000000000 P d68894b0170b4667bb9bb231b2a0c406 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:04:33.899201 5054 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d68894b0170b4667bb9bb231b2a0c406 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:04:33.899788 5057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:33.897866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.104:38683 (local address 127.0.0.1:45947)
0811 19:04:33.898012 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:33.898017 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:33.898220 (+ 203us) server_negotiation.cc:407] Connection header received
0811 19:04:33.898419 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:33.898422 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:33.898458 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:33.898561 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:33.899353 (+ 792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:04:33.899356 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:04:33.899359 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:04:33.899378 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:04:33.899405 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:04:33.899416 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:04:33.899511 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:33.899585 (+ 74us) server_negotiation.cc:299] Negotiation successful
0811 19:04:33.899640 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 19:04:33.900524 5054 raft_consensus.cc:504] T 00000000000000000000000000000000 P d68894b0170b4667bb9bb231b2a0c406 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d68894b0170b4667bb9bb231b2a0c406" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45947 } }
I20260811 19:04:33.900965 5054 leader_election.cc:302] T 00000000000000000000000000000000 P d68894b0170b4667bb9bb231b2a0c406 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d68894b0170b4667bb9bb231b2a0c406; no voters:
I20260811 19:04:33.901337 5054 leader_election.cc:288] T 00000000000000000000000000000000 P d68894b0170b4667bb9bb231b2a0c406 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:04:33.901466 5064 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d68894b0170b4667bb9bb231b2a0c406 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:04:33.901721 5064 raft_consensus.cc:686] T 00000000000000000000000000000000 P d68894b0170b4667bb9bb231b2a0c406 [term 1 LEADER]: Becoming Leader. State: Replica: d68894b0170b4667bb9bb231b2a0c406, State: Running, Role: LEADER
I20260811 19:04:33.902102 5064 consensus_queue.cc:237] T 00000000000000000000000000000000 P d68894b0170b4667bb9bb231b2a0c406 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d68894b0170b4667bb9bb231b2a0c406" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45947 } }
I20260811 19:04:33.902735 5054 sys_catalog.cc:564] T 00000000000000000000000000000000 P d68894b0170b4667bb9bb231b2a0c406 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:04:33.903764 5065 sys_catalog.cc:455] T 00000000000000000000000000000000 P d68894b0170b4667bb9bb231b2a0c406 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d68894b0170b4667bb9bb231b2a0c406" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d68894b0170b4667bb9bb231b2a0c406" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45947 } } }
I20260811 19:04:33.903898 5065 sys_catalog.cc:458] T 00000000000000000000000000000000 P d68894b0170b4667bb9bb231b2a0c406 [sys.catalog]: This master's current role is: LEADER
I20260811 19:04:33.904091 5066 sys_catalog.cc:455] T 00000000000000000000000000000000 P d68894b0170b4667bb9bb231b2a0c406 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d68894b0170b4667bb9bb231b2a0c406. Latest consensus state: current_term: 1 leader_uuid: "d68894b0170b4667bb9bb231b2a0c406" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d68894b0170b4667bb9bb231b2a0c406" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45947 } } }
I20260811 19:04:33.904244 5066 sys_catalog.cc:458] T 00000000000000000000000000000000 P d68894b0170b4667bb9bb231b2a0c406 [sys.catalog]: This master's current role is: LEADER
I20260811 19:04:33.905448 5074 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:04:33.908542 5074 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:04:33.914286 5074 catalog_manager.cc:1347] Generated new cluster ID: 72ca8af6532c4e56888a753f975ee0c6
I20260811 19:04:33.914372 5074 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:04:33.933354 5074 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:04:33.933715 5074 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:04:33.939992 5074 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d68894b0170b4667bb9bb231b2a0c406: Generated new TSK 0
I20260811 19:04:33.940824 5074 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 19:04:33.946312 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 4977
2026-08-11T19:04:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1491 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474792496492-13748-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:04:34 dist-test-slave-0kls krb5kdc[5089](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:04:34 dist-test-slave-0kls krb5kdc[5089](info): set up 2 sockets
Aug 11 19:04:34 dist-test-slave-0kls krb5kdc[5089](info): commencing operation
krb5kdc: starting...
W20260811 19:04:35.038921 13748 mini_kdc.cc:121] Time spent starting KDC: real 1.069s user 0.006s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474792496492-13748-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474792496492-13748-0/krb5kdc/test-user.keytab.
Aug 11 19:04:35 dist-test-slave-0kls krb5kdc[5089](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475075, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:04:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:04:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.109.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.109.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.109.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab.
Entry for principal kudu/127.13.109.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab.
I20260811 19:04:35.184948 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474792496492-13748-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474792496492-13748-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474792496492-13748-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.62:40147
--webserver_interface=127.13.109.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab
--principal=kudu/127.13.109.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:35987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.109.62:40147
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:04:35.304737 5104 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:04:35.305086 5104 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:04:35.305173 5104 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:04:35.309439 5104 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:04:35.309568 5104 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:04:35.309604 5104 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:04:35.309651 5104 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:04:35.309713 5104 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:04:35.315675 5104 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:35987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474792496492-13748-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474792496492-13748-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.109.62:40147
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab
--principal=kudu/127.13.109.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.62:40147
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.109.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5104
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474792496492-13748-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:04:35.316941 5104 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:04:35.318123 5104 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:04:35.327248 5109 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:04:35.327229 5112 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:04:35.327219 5110 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:04:35.327680 5104 server_base.cc:1034] running on GCE node
I20260811 19:04:35.328336 5104 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:04:35.329620 5104 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:04:35.330809 5104 hybrid_clock.cc:648] HybridClock initialized: now 1786475075330771 us; error 51 us; skew 500 ppm
Aug 11 19:04:35 dist-test-slave-0kls krb5kdc[5089](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475075, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:04:35.333935 5104 init.cc:377] Logged in from keytab as kudu/127.13.109.62@KRBTEST.COM (short username kudu)
I20260811 19:04:35.334959 5104 webserver.cc:460] Webserver started at http://127.13.109.62:35177/ using document root <none> and password file <none>
I20260811 19:04:35.335562 5104 fs_manager.cc:359] Metadata directory not provided
I20260811 19:04:35.335628 5104 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:04:35.335879 5104 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:04:35.338140 5104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474792496492-13748-0/minicluster-data/master-0/data/instance:
uuid: "e89785dcdc214c0b9f89e27719ea41d7"
format_stamp: "Formatted at 2026-08-11 19:04:35 on dist-test-slave-0kls"
server_key: "9a81ffcdddbe10fb1608b30c7eda04db"
server_key_iv: "a75fc8bff1a59cd4bf8bf650f46e4ccb"
server_key_version: "encryptionkey@0"
I20260811 19:04:35.338738 5104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474792496492-13748-0/minicluster-data/master-0/wal/instance:
uuid: "e89785dcdc214c0b9f89e27719ea41d7"
format_stamp: "Formatted at 2026-08-11 19:04:35 on dist-test-slave-0kls"
server_key: "9a81ffcdddbe10fb1608b30c7eda04db"
server_key_iv: "a75fc8bff1a59cd4bf8bf650f46e4ccb"
server_key_version: "encryptionkey@0"
I20260811 19:04:35.342628 5104 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 19:04:35.345140 5120 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:04:35.346491 5104 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 19:04:35.346639 5104 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474792496492-13748-0/minicluster-data/master-0/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474792496492-13748-0/minicluster-data/master-0/wal
uuid: "e89785dcdc214c0b9f89e27719ea41d7"
format_stamp: "Formatted at 2026-08-11 19:04:35 on dist-test-slave-0kls"
server_key: "9a81ffcdddbe10fb1608b30c7eda04db"
server_key_iv: "a75fc8bff1a59cd4bf8bf650f46e4ccb"
server_key_version: "encryptionkey@0"
I20260811 19:04:35.346760 5104 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474792496492-13748-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474792496492-13748-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474792496492-13748-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:04:35.357954 5104 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:04:35.360891 5104 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:04:35.361119 5104 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:04:35.368955 5104 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.62:40147
I20260811 19:04:35.368993 5178 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.62:40147 every 8 connection(s)
I20260811 19:04:35.370044 5104 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
I20260811 19:04:35.372973 5179 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:04:35.373122 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 5104
I20260811 19:04:35.373288 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474792496492-13748-0/minicluster-data/master-0/wal/instance
I20260811 19:04:35.373623 13748 external_mini_cluster.cc:1428] Setting key b0abd5e7f7943ad13c22992654f02ef1
I20260811 19:04:35.379979 5179 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e89785dcdc214c0b9f89e27719ea41d7: Bootstrap starting.
I20260811 19:04:35.382923 5179 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e89785dcdc214c0b9f89e27719ea41d7: Neither blocks nor log segments found. Creating new log.
Aug 11 19:04:35 dist-test-slave-0kls krb5kdc[5089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475075, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
I20260811 19:04:35.384004 5179 log.cc:824] T 00000000000000000000000000000000 P e89785dcdc214c0b9f89e27719ea41d7: Log is configured to *not* fsync() on all Append() calls
I20260811 19:04:35.386579 5179 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e89785dcdc214c0b9f89e27719ea41d7: No bootstrap required, opened a new log
I20260811 19:04:35.389762 5182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:35.375173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43703 (local address 127.13.109.62:40147)
0811 19:04:35.375650 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:35.375666 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:35.375712 (+ 46us) server_negotiation.cc:407] Connection header received
0811 19:04:35.376392 (+ 680us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:35.376409 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:35.376776 (+ 367us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:35.377134 (+ 358us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:35.378086 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:35.379112 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:35.380060 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:35.380629 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:35.380817 (+ 188us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:35.384004 (+ 3187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:04:35.384010 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:04:35.384027 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:04:35.384062 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:04:35.386235 (+ 2173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:35.386995 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:35.387002 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:35.387008 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:35.387147 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:35.387543 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:35.387551 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:35.387552 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:35.388026 (+ 474us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:04:35.388301 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:35.388629 (+ 328us) server_negotiation.cc:299] Negotiation successful
0811 19:04:35.388894 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331,"thread_start_us":142,"threads_started":1}
I20260811 19:04:35.390894 5179 raft_consensus.cc:348] T 00000000000000000000000000000000 P e89785dcdc214c0b9f89e27719ea41d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e89785dcdc214c0b9f89e27719ea41d7" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 40147 } }
I20260811 19:04:35.391206 5179 raft_consensus.cc:374] T 00000000000000000000000000000000 P e89785dcdc214c0b9f89e27719ea41d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:04:35.391294 5179 raft_consensus.cc:729] T 00000000000000000000000000000000 P e89785dcdc214c0b9f89e27719ea41d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e89785dcdc214c0b9f89e27719ea41d7, State: Initialized, Role: FOLLOWER
I20260811 19:04:35.391795 5179 consensus_queue.cc:260] T 00000000000000000000000000000000 P e89785dcdc214c0b9f89e27719ea41d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e89785dcdc214c0b9f89e27719ea41d7" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 40147 } }
I20260811 19:04:35.391979 5179 raft_consensus.cc:388] T 00000000000000000000000000000000 P e89785dcdc214c0b9f89e27719ea41d7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:04:35.392059 5179 raft_consensus.cc:482] T 00000000000000000000000000000000 P e89785dcdc214c0b9f89e27719ea41d7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:04:35.392206 5179 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e89785dcdc214c0b9f89e27719ea41d7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:04:35.393419 5179 raft_consensus.cc:504] T 00000000000000000000000000000000 P e89785dcdc214c0b9f89e27719ea41d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e89785dcdc214c0b9f89e27719ea41d7" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 40147 } }
I20260811 19:04:35.393651 13748 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 19:04:35.393898 5179 leader_election.cc:302] T 00000000000000000000000000000000 P e89785dcdc214c0b9f89e27719ea41d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e89785dcdc214c0b9f89e27719ea41d7; no voters:
I20260811 19:04:35.394342 5179 leader_election.cc:288] T 00000000000000000000000000000000 P e89785dcdc214c0b9f89e27719ea41d7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:04:35.394469 5185 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e89785dcdc214c0b9f89e27719ea41d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:04:35.394732 5185 raft_consensus.cc:686] T 00000000000000000000000000000000 P e89785dcdc214c0b9f89e27719ea41d7 [term 1 LEADER]: Becoming Leader. State: Replica: e89785dcdc214c0b9f89e27719ea41d7, State: Running, Role: LEADER
I20260811 19:04:35.395169 5185 consensus_queue.cc:237] T 00000000000000000000000000000000 P e89785dcdc214c0b9f89e27719ea41d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e89785dcdc214c0b9f89e27719ea41d7" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 40147 } }
I20260811 19:04:35.395958 5179 sys_catalog.cc:564] T 00000000000000000000000000000000 P e89785dcdc214c0b9f89e27719ea41d7 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:04:35.397025 5186 sys_catalog.cc:455] T 00000000000000000000000000000000 P e89785dcdc214c0b9f89e27719ea41d7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e89785dcdc214c0b9f89e27719ea41d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e89785dcdc214c0b9f89e27719ea41d7" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 40147 } } }
I20260811 19:04:35.397177 5186 sys_catalog.cc:458] T 00000000000000000000000000000000 P e89785dcdc214c0b9f89e27719ea41d7 [sys.catalog]: This master's current role is: LEADER
I20260811 19:04:35.397541 5187 sys_catalog.cc:455] T 00000000000000000000000000000000 P e89785dcdc214c0b9f89e27719ea41d7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e89785dcdc214c0b9f89e27719ea41d7. Latest consensus state: current_term: 1 leader_uuid: "e89785dcdc214c0b9f89e27719ea41d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e89785dcdc214c0b9f89e27719ea41d7" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 40147 } } }
I20260811 19:04:35.397599 5202 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:04:35.397653 5187 sys_catalog.cc:458] T 00000000000000000000000000000000 P e89785dcdc214c0b9f89e27719ea41d7 [sys.catalog]: This master's current role is: LEADER
I20260811 19:04:35.401635 5202 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:04:35.407662 5182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:35.396589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40409 (local address 127.13.109.62:40147)
0811 19:04:35.396715 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:35.396722 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:35.396740 (+ 18us) server_negotiation.cc:407] Connection header received
0811 19:04:35.396791 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:35.396795 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:35.396855 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:35.396957 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:35.397986 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:35.398785 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:35.400126 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:35.400466 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:35.400581 (+ 115us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:35.402955 (+ 2374us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:04:35.402960 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:04:35.402963 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:04:35.402999 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:04:35.404859 (+ 1860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:35.405526 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:35.405530 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:35.405531 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:35.405640 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:35.406124 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:35.406128 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:35.406130 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:35.406383 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:04:35.406566 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:35.407361 (+ 795us) server_negotiation.cc:299] Negotiation successful
0811 19:04:35.407480 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 19:04:35.408929 5202 catalog_manager.cc:1347] Generated new cluster ID: 3d779dce3ee04f70801ec2b6a91e1c19
I20260811 19:04:35.409019 5202 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:04:35.429451 5202 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:04:35.430589 5202 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:04:35.438679 5202 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e89785dcdc214c0b9f89e27719ea41d7: Generated new TSK 0
I20260811 19:04:35.439635 5202 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 19:04:35.448266 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 5104
W20260811 19:04:35.458986 5087 connection.cc:535] client connection to 127.13.109.62:40147 recv error: Network error: failed to read from TLS socket (remote: 127.13.109.62:40147): Connection reset by peer (error 104)
2026-08-11T19:04:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1503 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474792496492-13748-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:04:35 dist-test-slave-0kls krb5kdc[5214](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:04:35 dist-test-slave-0kls krb5kdc[5214](info): set up 2 sockets
Aug 11 19:04:35 dist-test-slave-0kls krb5kdc[5214](info): commencing operation
krb5kdc: starting...
W20260811 19:04:36.558969 13748 mini_kdc.cc:121] Time spent starting KDC: real 1.085s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474792496492-13748-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474792496492-13748-0/krb5kdc/test-user.keytab.
Aug 11 19:04:36 dist-test-slave-0kls krb5kdc[5214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475076, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:04:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:04:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.109.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.109.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.109.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab.
Entry for principal kudu/127.13.109.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab.
I20260811 19:04:36.692996 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474792496492-13748-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474792496492-13748-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474792496492-13748-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.62:42831
--webserver_interface=127.13.109.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab
--principal=kudu/127.13.109.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:35563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.109.62:42831
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:04:36.817112 5231 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:04:36.817409 5231 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:04:36.817471 5231 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:04:36.821931 5231 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:04:36.822001 5231 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:04:36.822022 5231 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:04:36.822041 5231 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:04:36.822057 5231 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:04:36.827719 5231 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:35563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474792496492-13748-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474792496492-13748-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.109.62:42831
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab
--principal=kudu/127.13.109.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.62:42831
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.109.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5231
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474792496492-13748-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:04:36.828966 5231 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:04:36.830147 5231 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 19:04:36.839401 5231 server_base.cc:1034] running on GCE node
W20260811 19:04:36.839358 5238 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:04:36.839377 5240 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:04:36.839358 5237 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:04:36.840324 5231 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:04:36.841548 5231 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:04:36.842746 5231 hybrid_clock.cc:648] HybridClock initialized: now 1786475076842727 us; error 32 us; skew 500 ppm
Aug 11 19:04:36 dist-test-slave-0kls krb5kdc[5214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475076, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:04:36.845665 5231 init.cc:377] Logged in from keytab as kudu/127.13.109.62@KRBTEST.COM (short username kudu)
I20260811 19:04:36.846679 5231 webserver.cc:460] Webserver started at http://127.13.109.62:35933/ using document root <none> and password file <none>
I20260811 19:04:36.847282 5231 fs_manager.cc:359] Metadata directory not provided
I20260811 19:04:36.847347 5231 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:04:36.847573 5231 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:04:36.849682 5231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474792496492-13748-0/minicluster-data/master-0/data/instance:
uuid: "ffaf24c2e6b74ae6af4c8871fd94df89"
format_stamp: "Formatted at 2026-08-11 19:04:36 on dist-test-slave-0kls"
server_key: "3955240e09ca11ef4b110f560ad73a26"
server_key_iv: "b49dd1cbe68b61b7280b7693a91afb26"
server_key_version: "encryptionkey@0"
I20260811 19:04:36.850335 5231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474792496492-13748-0/minicluster-data/master-0/wal/instance:
uuid: "ffaf24c2e6b74ae6af4c8871fd94df89"
format_stamp: "Formatted at 2026-08-11 19:04:36 on dist-test-slave-0kls"
server_key: "3955240e09ca11ef4b110f560ad73a26"
server_key_iv: "b49dd1cbe68b61b7280b7693a91afb26"
server_key_version: "encryptionkey@0"
I20260811 19:04:36.854290 5231 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 19:04:36.856911 5247 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:04:36.858399 5231 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 19:04:36.858553 5231 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474792496492-13748-0/minicluster-data/master-0/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474792496492-13748-0/minicluster-data/master-0/wal
uuid: "ffaf24c2e6b74ae6af4c8871fd94df89"
format_stamp: "Formatted at 2026-08-11 19:04:36 on dist-test-slave-0kls"
server_key: "3955240e09ca11ef4b110f560ad73a26"
server_key_iv: "b49dd1cbe68b61b7280b7693a91afb26"
server_key_version: "encryptionkey@0"
I20260811 19:04:36.858685 5231 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474792496492-13748-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474792496492-13748-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474792496492-13748-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:04:36.874802 5231 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:04:36.878551 5231 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:04:36.878787 5231 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:04:36.886616 5231 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.62:42831
I20260811 19:04:36.886655 5308 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.62:42831 every 8 connection(s)
I20260811 19:04:36.887676 5231 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
I20260811 19:04:36.890677 5309 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:04:36.891374 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 5231
I20260811 19:04:36.891523 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474792496492-13748-0/minicluster-data/master-0/wal/instance
I20260811 19:04:36.891842 13748 external_mini_cluster.cc:1428] Setting key 137f0e2423e03bc5613b257c20fd100c
I20260811 19:04:36.898135 5309 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ffaf24c2e6b74ae6af4c8871fd94df89: Bootstrap starting.
I20260811 19:04:36.900995 5309 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ffaf24c2e6b74ae6af4c8871fd94df89: Neither blocks nor log segments found. Creating new log.
I20260811 19:04:36.901999 5309 log.cc:824] T 00000000000000000000000000000000 P ffaf24c2e6b74ae6af4c8871fd94df89: Log is configured to *not* fsync() on all Append() calls
I20260811 19:04:36.902761 5312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:36.893377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60693 (local address 127.13.109.62:42831)
0811 19:04:36.893911 (+ 534us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:36.893933 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:36.893984 (+ 51us) server_negotiation.cc:407] Connection header received
0811 19:04:36.894857 (+ 873us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:36.894881 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:36.895327 (+ 446us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:36.895704 (+ 377us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:36.896128 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:36.897716 (+ 1588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:36.898781 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:36.899496 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:36.899669 (+ 173us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:36.900617 (+ 948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:04:36.900623 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:04:36.900639 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:04:36.901196 (+ 557us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:04:36.901275 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:04:36.901311 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:04:36.901447 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:36.901644 (+ 197us) server_negotiation.cc:299] Negotiation successful
0811 19:04:36.901848 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":136,"threads_started":1}
I20260811 19:04:36.904323 5309 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ffaf24c2e6b74ae6af4c8871fd94df89: No bootstrap required, opened a new log
I20260811 19:04:36.905988 13748 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 19:04:36.908712 5309 raft_consensus.cc:348] T 00000000000000000000000000000000 P ffaf24c2e6b74ae6af4c8871fd94df89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffaf24c2e6b74ae6af4c8871fd94df89" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 42831 } }
I20260811 19:04:36.909030 5309 raft_consensus.cc:374] T 00000000000000000000000000000000 P ffaf24c2e6b74ae6af4c8871fd94df89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:04:36.909104 5309 raft_consensus.cc:729] T 00000000000000000000000000000000 P ffaf24c2e6b74ae6af4c8871fd94df89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffaf24c2e6b74ae6af4c8871fd94df89, State: Initialized, Role: FOLLOWER
I20260811 19:04:36.909551 5309 consensus_queue.cc:260] T 00000000000000000000000000000000 P ffaf24c2e6b74ae6af4c8871fd94df89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffaf24c2e6b74ae6af4c8871fd94df89" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 42831 } }
I20260811 19:04:36.909678 5309 raft_consensus.cc:388] T 00000000000000000000000000000000 P ffaf24c2e6b74ae6af4c8871fd94df89 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:04:36.909771 5309 raft_consensus.cc:482] T 00000000000000000000000000000000 P ffaf24c2e6b74ae6af4c8871fd94df89 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:04:36.909950 5309 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ffaf24c2e6b74ae6af4c8871fd94df89 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:04:36.910952 5309 raft_consensus.cc:504] T 00000000000000000000000000000000 P ffaf24c2e6b74ae6af4c8871fd94df89 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffaf24c2e6b74ae6af4c8871fd94df89" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 42831 } }
I20260811 19:04:36.911334 5309 leader_election.cc:302] T 00000000000000000000000000000000 P ffaf24c2e6b74ae6af4c8871fd94df89 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ffaf24c2e6b74ae6af4c8871fd94df89; no voters:
I20260811 19:04:36.911692 5309 leader_election.cc:288] T 00000000000000000000000000000000 P ffaf24c2e6b74ae6af4c8871fd94df89 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:04:36.911787 5320 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ffaf24c2e6b74ae6af4c8871fd94df89 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:04:36.911962 5320 raft_consensus.cc:686] T 00000000000000000000000000000000 P ffaf24c2e6b74ae6af4c8871fd94df89 [term 1 LEADER]: Becoming Leader. State: Replica: ffaf24c2e6b74ae6af4c8871fd94df89, State: Running, Role: LEADER
I20260811 19:04:36.912333 5320 consensus_queue.cc:237] T 00000000000000000000000000000000 P ffaf24c2e6b74ae6af4c8871fd94df89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffaf24c2e6b74ae6af4c8871fd94df89" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 42831 } }
I20260811 19:04:36.912530 5312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:36.908084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47999 (local address 127.13.109.62:42831)
0811 19:04:36.908228 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:36.908233 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:36.908461 (+ 228us) server_negotiation.cc:407] Connection header received
0811 19:04:36.908582 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:36.908584 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:36.908626 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:36.908691 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:36.909031 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:36.909735 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:36.910649 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:36.910909 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:36.911020 (+ 111us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:36.911978 (+ 958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:04:36.911984 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:04:36.911988 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:04:36.912004 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:04:36.912034 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:04:36.912049 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:04:36.912137 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:36.912343 (+ 206us) server_negotiation.cc:299] Negotiation successful
0811 19:04:36.912382 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 19:04:36.913138 5309 sys_catalog.cc:564] T 00000000000000000000000000000000 P ffaf24c2e6b74ae6af4c8871fd94df89 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:04:36.914129 5323 sys_catalog.cc:455] T 00000000000000000000000000000000 P ffaf24c2e6b74ae6af4c8871fd94df89 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ffaf24c2e6b74ae6af4c8871fd94df89" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffaf24c2e6b74ae6af4c8871fd94df89" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 42831 } } }
I20260811 19:04:36.914343 5323 sys_catalog.cc:458] T 00000000000000000000000000000000 P ffaf24c2e6b74ae6af4c8871fd94df89 [sys.catalog]: This master's current role is: LEADER
I20260811 19:04:36.914628 5323 sys_catalog.cc:455] T 00000000000000000000000000000000 P ffaf24c2e6b74ae6af4c8871fd94df89 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ffaf24c2e6b74ae6af4c8871fd94df89. Latest consensus state: current_term: 1 leader_uuid: "ffaf24c2e6b74ae6af4c8871fd94df89" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffaf24c2e6b74ae6af4c8871fd94df89" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 42831 } } }
I20260811 19:04:36.914778 5323 sys_catalog.cc:458] T 00000000000000000000000000000000 P ffaf24c2e6b74ae6af4c8871fd94df89 [sys.catalog]: This master's current role is: LEADER
I20260811 19:04:36.914794 5332 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:04:36.918311 5332 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:04:36.924738 5332 catalog_manager.cc:1347] Generated new cluster ID: dc1f0d1ef3ac4c0cb7c7bb93551335ef
I20260811 19:04:36.924840 5332 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:04:36.944629 5332 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:04:36.945709 5332 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:04:36.952234 5332 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ffaf24c2e6b74ae6af4c8871fd94df89: Generated new TSK 0
I20260811 19:04:36.953250 5332 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 19:04:36.960732 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 5231
W20260811 19:04:36.970454 5212 connection.cc:535] client connection to 127.13.109.62:42831 recv error: Network error: failed to read from TLS socket (remote: 127.13.109.62:42831): Connection reset by peer (error 104)
2026-08-11T19:04:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1511 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474792496492-13748-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:04:37 dist-test-slave-0kls krb5kdc[5345](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:04:37 dist-test-slave-0kls krb5kdc[5345](info): set up 2 sockets
Aug 11 19:04:37 dist-test-slave-0kls krb5kdc[5345](info): commencing operation
krb5kdc: starting...
W20260811 19:04:38.049612 13748 mini_kdc.cc:121] Time spent starting KDC: real 1.064s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474792496492-13748-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474792496492-13748-0/krb5kdc/test-user.keytab.
Aug 11 19:04:38 dist-test-slave-0kls krb5kdc[5345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475078, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:04:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:04:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.109.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.109.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.109.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab.
Entry for principal kudu/127.13.109.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab.
I20260811 19:04:38.187400 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474792496492-13748-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474792496492-13748-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474792496492-13748-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.62:39409
--webserver_interface=127.13.109.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab
--principal=kudu/127.13.109.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:33987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.109.62:39409
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:04:38.307149 5360 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:04:38.307498 5360 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:04:38.307545 5360 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:04:38.311880 5360 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:04:38.311955 5360 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:04:38.311976 5360 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:04:38.311993 5360 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:04:38.312011 5360 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:04:38.317998 5360 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:33987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474792496492-13748-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474792496492-13748-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.109.62:39409
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab
--principal=kudu/127.13.109.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.62:39409
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.109.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5360
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474792496492-13748-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:04:38.319244 5360 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:04:38.320384 5360 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:04:38.329917 5366 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:04:38.329903 5367 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:04:38.329903 5369 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:04:38.330044 5360 server_base.cc:1034] running on GCE node
I20260811 19:04:38.330916 5360 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:04:38.332150 5360 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:04:38.333348 5360 hybrid_clock.cc:648] HybridClock initialized: now 1786475078333327 us; error 37 us; skew 500 ppm
Aug 11 19:04:38 dist-test-slave-0kls krb5kdc[5345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475078, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:04:38.341567 5360 init.cc:377] Logged in from keytab as kudu/127.13.109.62@KRBTEST.COM (short username kudu)
I20260811 19:04:38.342765 5360 webserver.cc:460] Webserver started at http://127.13.109.62:38187/ using document root <none> and password file <none>
I20260811 19:04:38.343456 5360 fs_manager.cc:359] Metadata directory not provided
I20260811 19:04:38.343571 5360 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:04:38.343847 5360 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:04:38.346300 5360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474792496492-13748-0/minicluster-data/master-0/data/instance:
uuid: "79b1357c1ff0472c916d6af7e77ff5ea"
format_stamp: "Formatted at 2026-08-11 19:04:38 on dist-test-slave-0kls"
server_key: "0be7456093cef1d9c9d7128ae1dbbf2d"
server_key_iv: "1f3027e5ab516fa3cd88aa53496521d7"
server_key_version: "encryptionkey@0"
I20260811 19:04:38.346943 5360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474792496492-13748-0/minicluster-data/master-0/wal/instance:
uuid: "79b1357c1ff0472c916d6af7e77ff5ea"
format_stamp: "Formatted at 2026-08-11 19:04:38 on dist-test-slave-0kls"
server_key: "0be7456093cef1d9c9d7128ae1dbbf2d"
server_key_iv: "1f3027e5ab516fa3cd88aa53496521d7"
server_key_version: "encryptionkey@0"
I20260811 19:04:38.351191 5360 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260811 19:04:38.354064 5377 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:04:38.355456 5360 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 19:04:38.355623 5360 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474792496492-13748-0/minicluster-data/master-0/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474792496492-13748-0/minicluster-data/master-0/wal
uuid: "79b1357c1ff0472c916d6af7e77ff5ea"
format_stamp: "Formatted at 2026-08-11 19:04:38 on dist-test-slave-0kls"
server_key: "0be7456093cef1d9c9d7128ae1dbbf2d"
server_key_iv: "1f3027e5ab516fa3cd88aa53496521d7"
server_key_version: "encryptionkey@0"
I20260811 19:04:38.355765 5360 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474792496492-13748-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474792496492-13748-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474792496492-13748-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:04:38.360898 5360 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:04:38.364553 5360 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:04:38.364782 5360 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:04:38.372700 5360 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.62:39409
I20260811 19:04:38.372730 5433 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.62:39409 every 8 connection(s)
I20260811 19:04:38.373703 5360 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
I20260811 19:04:38.375581 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 5360
I20260811 19:04:38.375754 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474792496492-13748-0/minicluster-data/master-0/wal/instance
I20260811 19:04:38.376080 13748 external_mini_cluster.cc:1428] Setting key 21cd6f4ab9e4dbf3e3fd38a0cbf19507
I20260811 19:04:38.376992 5434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:04:38.381383 5437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:38.377556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54511 (local address 127.13.109.62:39409)
0811 19:04:38.378044 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:38.378060 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:38.378096 (+ 36us) server_negotiation.cc:407] Connection header received
0811 19:04:38.378754 (+ 658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:38.378769 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:38.379068 (+ 299us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:38.379409 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:38.380210 (+ 801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:04:38.380215 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:04:38.380226 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:04:38.380236 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:04:38.380289 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:04:38.380308 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:04:38.380464 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:38.380533 (+ 69us) server_negotiation.cc:299] Negotiation successful
0811 19:04:38.380695 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":179,"threads_started":1}
I20260811 19:04:38.383606 5434 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79b1357c1ff0472c916d6af7e77ff5ea: Bootstrap starting.
I20260811 19:04:38.384438 13748 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 19:04:38.386270 5434 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 79b1357c1ff0472c916d6af7e77ff5ea: Neither blocks nor log segments found. Creating new log.
I20260811 19:04:38.387303 5434 log.cc:824] T 00000000000000000000000000000000 P 79b1357c1ff0472c916d6af7e77ff5ea: Log is configured to *not* fsync() on all Append() calls
I20260811 19:04:38.388417 5437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:38.386614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41247 (local address 127.13.109.62:39409)
0811 19:04:38.386761 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:38.386766 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:38.386932 (+ 166us) server_negotiation.cc:407] Connection header received
0811 19:04:38.387043 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:38.387049 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:38.387096 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:38.387185 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:38.387968 (+ 783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:04:38.387971 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:04:38.387973 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:04:38.387984 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:04:38.388012 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:04:38.388022 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:04:38.388094 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:38.388216 (+ 122us) server_negotiation.cc:299] Negotiation successful
0811 19:04:38.388252 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 19:04:38.390023 5434 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79b1357c1ff0472c916d6af7e77ff5ea: No bootstrap required, opened a new log
I20260811 19:04:38.393898 5434 raft_consensus.cc:348] T 00000000000000000000000000000000 P 79b1357c1ff0472c916d6af7e77ff5ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79b1357c1ff0472c916d6af7e77ff5ea" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 39409 } }
I20260811 19:04:38.394151 5434 raft_consensus.cc:374] T 00000000000000000000000000000000 P 79b1357c1ff0472c916d6af7e77ff5ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:04:38.394238 5434 raft_consensus.cc:729] T 00000000000000000000000000000000 P 79b1357c1ff0472c916d6af7e77ff5ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79b1357c1ff0472c916d6af7e77ff5ea, State: Initialized, Role: FOLLOWER
I20260811 19:04:38.394697 5434 consensus_queue.cc:260] T 00000000000000000000000000000000 P 79b1357c1ff0472c916d6af7e77ff5ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79b1357c1ff0472c916d6af7e77ff5ea" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 39409 } }
I20260811 19:04:38.394878 5434 raft_consensus.cc:388] T 00000000000000000000000000000000 P 79b1357c1ff0472c916d6af7e77ff5ea [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:04:38.394950 5434 raft_consensus.cc:482] T 00000000000000000000000000000000 P 79b1357c1ff0472c916d6af7e77ff5ea [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:04:38.395058 5434 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 79b1357c1ff0472c916d6af7e77ff5ea [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:04:38.396274 5434 raft_consensus.cc:504] T 00000000000000000000000000000000 P 79b1357c1ff0472c916d6af7e77ff5ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79b1357c1ff0472c916d6af7e77ff5ea" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 39409 } }
I20260811 19:04:38.396639 5434 leader_election.cc:302] T 00000000000000000000000000000000 P 79b1357c1ff0472c916d6af7e77ff5ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 79b1357c1ff0472c916d6af7e77ff5ea; no voters:
I20260811 19:04:38.396991 5434 leader_election.cc:288] T 00000000000000000000000000000000 P 79b1357c1ff0472c916d6af7e77ff5ea [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:04:38.397145 5445 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 79b1357c1ff0472c916d6af7e77ff5ea [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:04:38.397403 5445 raft_consensus.cc:686] T 00000000000000000000000000000000 P 79b1357c1ff0472c916d6af7e77ff5ea [term 1 LEADER]: Becoming Leader. State: Replica: 79b1357c1ff0472c916d6af7e77ff5ea, State: Running, Role: LEADER
I20260811 19:04:38.397773 5445 consensus_queue.cc:237] T 00000000000000000000000000000000 P 79b1357c1ff0472c916d6af7e77ff5ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79b1357c1ff0472c916d6af7e77ff5ea" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 39409 } }
I20260811 19:04:38.398447 5434 sys_catalog.cc:564] T 00000000000000000000000000000000 P 79b1357c1ff0472c916d6af7e77ff5ea [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:04:38.399914 5447 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79b1357c1ff0472c916d6af7e77ff5ea [sys.catalog]: SysCatalogTable state changed. Reason: New leader 79b1357c1ff0472c916d6af7e77ff5ea. Latest consensus state: current_term: 1 leader_uuid: "79b1357c1ff0472c916d6af7e77ff5ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79b1357c1ff0472c916d6af7e77ff5ea" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 39409 } } }
I20260811 19:04:38.400063 5447 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79b1357c1ff0472c916d6af7e77ff5ea [sys.catalog]: This master's current role is: LEADER
I20260811 19:04:38.399878 5446 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79b1357c1ff0472c916d6af7e77ff5ea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "79b1357c1ff0472c916d6af7e77ff5ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79b1357c1ff0472c916d6af7e77ff5ea" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 39409 } } }
I20260811 19:04:38.400427 5446 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79b1357c1ff0472c916d6af7e77ff5ea [sys.catalog]: This master's current role is: LEADER
I20260811 19:04:38.400656 5455 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:04:38.404083 5455 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:04:38.410089 5455 catalog_manager.cc:1347] Generated new cluster ID: f2981e50be8c4fbe9774ccdbfcdce6d3
I20260811 19:04:38.410178 5455 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:04:38.437963 5455 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:04:38.438380 5455 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:04:38.450547 5455 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 79b1357c1ff0472c916d6af7e77ff5ea: Generated new TSK 0
I20260811 19:04:38.451706 5455 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 19:04:38.460225 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 5360
2026-08-11T19:04:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1498 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474792496492-13748-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:04:38 dist-test-slave-0kls krb5kdc[5467](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:04:38 dist-test-slave-0kls krb5kdc[5467](info): set up 2 sockets
Aug 11 19:04:38 dist-test-slave-0kls krb5kdc[5467](info): commencing operation
krb5kdc: starting...
W20260811 19:04:39.554497 13748 mini_kdc.cc:121] Time spent starting KDC: real 1.070s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474792496492-13748-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474792496492-13748-0/krb5kdc/test-user.keytab.
Aug 11 19:04:39 dist-test-slave-0kls krb5kdc[5467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475079, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:04:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:04:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.109.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.109.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.109.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab.
Entry for principal kudu/127.13.109.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab.
I20260811 19:04:39.697278 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474792496492-13748-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474792496492-13748-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474792496492-13748-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.62:40391
--webserver_interface=127.13.109.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab
--principal=kudu/127.13.109.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:41949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.109.62:40391
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:04:39.820104 5483 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:04:39.820438 5483 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:04:39.820516 5483 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:04:39.824635 5483 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:04:39.824750 5483 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:04:39.824791 5483 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:04:39.824833 5483 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:04:39.824867 5483 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:04:39.830539 5483 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:41949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474792496492-13748-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474792496492-13748-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.109.62:40391
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab
--principal=kudu/127.13.109.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.62:40391
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.109.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5483
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474792496492-13748-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:04:39.831772 5483 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:04:39.832867 5483 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:04:39.842805 5489 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:04:39.842798 5490 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:04:39.842798 5492 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:04:39.842990 5483 server_base.cc:1034] running on GCE node
I20260811 19:04:39.843868 5483 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:04:39.845096 5483 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:04:39.846269 5483 hybrid_clock.cc:648] HybridClock initialized: now 1786475079846246 us; error 36 us; skew 500 ppm
Aug 11 19:04:39 dist-test-slave-0kls krb5kdc[5467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475079, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:04:39.849247 5483 init.cc:377] Logged in from keytab as kudu/127.13.109.62@KRBTEST.COM (short username kudu)
I20260811 19:04:39.850414 5483 webserver.cc:460] Webserver started at http://127.13.109.62:34205/ using document root <none> and password file <none>
I20260811 19:04:39.851063 5483 fs_manager.cc:359] Metadata directory not provided
I20260811 19:04:39.851167 5483 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:04:39.851429 5483 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:04:39.853715 5483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474792496492-13748-0/minicluster-data/master-0/data/instance:
uuid: "6f367a5294024c32b4a4c1c902017a3c"
format_stamp: "Formatted at 2026-08-11 19:04:39 on dist-test-slave-0kls"
server_key: "b796319e543819cd80ce139a20794228"
server_key_iv: "d696537ad6d2c35ddecc989e814fde15"
server_key_version: "encryptionkey@0"
I20260811 19:04:39.854486 5483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474792496492-13748-0/minicluster-data/master-0/wal/instance:
uuid: "6f367a5294024c32b4a4c1c902017a3c"
format_stamp: "Formatted at 2026-08-11 19:04:39 on dist-test-slave-0kls"
server_key: "b796319e543819cd80ce139a20794228"
server_key_iv: "d696537ad6d2c35ddecc989e814fde15"
server_key_version: "encryptionkey@0"
I20260811 19:04:39.858788 5483 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 19:04:39.861694 5501 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:04:39.863142 5483 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 19:04:39.863312 5483 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474792496492-13748-0/minicluster-data/master-0/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474792496492-13748-0/minicluster-data/master-0/wal
uuid: "6f367a5294024c32b4a4c1c902017a3c"
format_stamp: "Formatted at 2026-08-11 19:04:39 on dist-test-slave-0kls"
server_key: "b796319e543819cd80ce139a20794228"
server_key_iv: "d696537ad6d2c35ddecc989e814fde15"
server_key_version: "encryptionkey@0"
I20260811 19:04:39.863452 5483 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474792496492-13748-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474792496492-13748-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474792496492-13748-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:04:39.888826 5483 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:04:39.891928 5483 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:04:39.892187 5483 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:04:39.900024 5483 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.62:40391
I20260811 19:04:39.900050 5561 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.62:40391 every 8 connection(s)
I20260811 19:04:39.901069 5483 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
I20260811 19:04:39.903990 5562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:04:39.905992 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 5483
I20260811 19:04:39.906144 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474792496492-13748-0/minicluster-data/master-0/wal/instance
I20260811 19:04:39.906445 13748 external_mini_cluster.cc:1428] Setting key 9dbc1bb47e1233e7aae439b00a536802
I20260811 19:04:39.910961 5562 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f367a5294024c32b4a4c1c902017a3c: Bootstrap starting.
I20260811 19:04:39.913383 5562 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6f367a5294024c32b4a4c1c902017a3c: Neither blocks nor log segments found. Creating new log.
I20260811 19:04:39.914320 5562 log.cc:824] T 00000000000000000000000000000000 P 6f367a5294024c32b4a4c1c902017a3c: Log is configured to *not* fsync() on all Append() calls
Aug 11 19:04:39 dist-test-slave-0kls krb5kdc[5467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475079, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
I20260811 19:04:39.916816 5562 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f367a5294024c32b4a4c1c902017a3c: No bootstrap required, opened a new log
I20260811 19:04:39.920702 5562 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6f367a5294024c32b4a4c1c902017a3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f367a5294024c32b4a4c1c902017a3c" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 40391 } }
I20260811 19:04:39.920976 5565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:39.908011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51121 (local address 127.13.109.62:40391)
0811 19:04:39.908502 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:39.908517 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:39.908572 (+ 55us) server_negotiation.cc:407] Connection header received
0811 19:04:39.909239 (+ 667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:39.909257 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:39.909601 (+ 344us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:39.909988 (+ 387us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:39.910841 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:39.911878 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:39.912748 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:39.913223 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:39.913415 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:39.916036 (+ 2621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:04:39.916043 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:04:39.916059 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:04:39.916104 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:04:39.917965 (+ 1861us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:39.918423 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:39.918430 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:39.918442 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:39.918557 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:39.918859 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:39.918863 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:39.918865 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:39.919312 (+ 447us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:04:39.919520 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:39.919844 (+ 324us) server_negotiation.cc:299] Negotiation successful
0811 19:04:39.920175 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":172,"threads_started":1}
I20260811 19:04:39.920984 5562 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6f367a5294024c32b4a4c1c902017a3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:04:39.921172 5562 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6f367a5294024c32b4a4c1c902017a3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f367a5294024c32b4a4c1c902017a3c, State: Initialized, Role: FOLLOWER
I20260811 19:04:39.921644 5562 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6f367a5294024c32b4a4c1c902017a3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f367a5294024c32b4a4c1c902017a3c" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 40391 } }
I20260811 19:04:39.921852 5562 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6f367a5294024c32b4a4c1c902017a3c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:04:39.921929 5562 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6f367a5294024c32b4a4c1c902017a3c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:04:39.922065 5562 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6f367a5294024c32b4a4c1c902017a3c [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:04:39.923460 5562 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6f367a5294024c32b4a4c1c902017a3c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f367a5294024c32b4a4c1c902017a3c" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 40391 } }
I20260811 19:04:39.923893 5562 leader_election.cc:302] T 00000000000000000000000000000000 P 6f367a5294024c32b4a4c1c902017a3c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6f367a5294024c32b4a4c1c902017a3c; no voters:
I20260811 19:04:39.924296 5562 leader_election.cc:288] T 00000000000000000000000000000000 P 6f367a5294024c32b4a4c1c902017a3c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:04:39.924366 13748 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 19:04:39.924435 5567 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6f367a5294024c32b4a4c1c902017a3c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:04:39.924624 5567 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6f367a5294024c32b4a4c1c902017a3c [term 1 LEADER]: Becoming Leader. State: Replica: 6f367a5294024c32b4a4c1c902017a3c, State: Running, Role: LEADER
I20260811 19:04:39.924945 5567 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6f367a5294024c32b4a4c1c902017a3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f367a5294024c32b4a4c1c902017a3c" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 40391 } }
I20260811 19:04:39.925640 5562 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6f367a5294024c32b4a4c1c902017a3c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:04:39.926717 5568 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f367a5294024c32b4a4c1c902017a3c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6f367a5294024c32b4a4c1c902017a3c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f367a5294024c32b4a4c1c902017a3c" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 40391 } } }
I20260811 19:04:39.926875 5568 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f367a5294024c32b4a4c1c902017a3c [sys.catalog]: This master's current role is: LEADER
I20260811 19:04:39.927151 5568 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f367a5294024c32b4a4c1c902017a3c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6f367a5294024c32b4a4c1c902017a3c. Latest consensus state: current_term: 1 leader_uuid: "6f367a5294024c32b4a4c1c902017a3c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f367a5294024c32b4a4c1c902017a3c" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 40391 } } }
I20260811 19:04:39.927249 5568 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f367a5294024c32b4a4c1c902017a3c [sys.catalog]: This master's current role is: LEADER
I20260811 19:04:39.927281 5581 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:04:39.931100 5581 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:04:39.937105 5565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:39.927852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39647 (local address 127.13.109.62:40391)
0811 19:04:39.927975 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:39.927980 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:39.928285 (+ 305us) server_negotiation.cc:407] Connection header received
0811 19:04:39.928499 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:39.928502 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:39.928558 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:39.928654 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:39.929493 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:39.930346 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:39.931486 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:39.931832 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:39.931913 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:39.933824 (+ 1911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:04:39.933828 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:04:39.933830 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:04:39.933859 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:04:39.935374 (+ 1515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:39.935823 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:39.935827 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:39.935829 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:39.935919 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:39.936208 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:39.936214 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:39.936217 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:39.936413 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:04:39.936550 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:39.936814 (+ 264us) server_negotiation.cc:299] Negotiation successful
0811 19:04:39.936940 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 19:04:39.938915 5581 catalog_manager.cc:1347] Generated new cluster ID: 26c8887eef074c9398fc729a63f82a64
I20260811 19:04:39.938985 5581 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:04:39.955873 5581 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:04:39.957307 5581 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:04:39.963395 5581 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6f367a5294024c32b4a4c1c902017a3c: Generated new TSK 0
I20260811 19:04:39.964200 5581 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 19:04:39.968467 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 5483
2026-08-11T19:04:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1508 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474792496492-13748-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:04:40 dist-test-slave-0kls krb5kdc[5597](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:04:40 dist-test-slave-0kls krb5kdc[5597](info): set up 2 sockets
Aug 11 19:04:40 dist-test-slave-0kls krb5kdc[5597](info): commencing operation
krb5kdc: starting...
W20260811 19:04:41.066044 13748 mini_kdc.cc:121] Time spent starting KDC: real 1.073s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474792496492-13748-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474792496492-13748-0/krb5kdc/test-user.keytab.
Aug 11 19:04:41 dist-test-slave-0kls krb5kdc[5597](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475081, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:04:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:04:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.109.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.109.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.109.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab.
Entry for principal kudu/127.13.109.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab.
I20260811 19:04:41.206835 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474792496492-13748-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474792496492-13748-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474792496492-13748-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.62:43149
--webserver_interface=127.13.109.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab
--principal=kudu/127.13.109.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:44643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.109.62:43149
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:04:41.326331 5612 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:04:41.326673 5612 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:04:41.326764 5612 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:04:41.330991 5612 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:04:41.331092 5612 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:04:41.331159 5612 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:04:41.331211 5612 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:04:41.331251 5612 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:04:41.336791 5612 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:44643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474792496492-13748-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474792496492-13748-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.109.62:43149
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab
--principal=kudu/127.13.109.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.62:43149
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.109.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5612
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474792496492-13748-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:04:41.338014 5612 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:04:41.339102 5612 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:04:41.348042 5618 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:04:41.348074 5620 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:04:41.348073 5617 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:04:41.348347 5612 server_base.cc:1034] running on GCE node
I20260811 19:04:41.349077 5612 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:04:41.350371 5612 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:04:41.351553 5612 hybrid_clock.cc:648] HybridClock initialized: now 1786475081351526 us; error 40 us; skew 500 ppm
Aug 11 19:04:41 dist-test-slave-0kls krb5kdc[5597](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475081, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:04:41.354468 5612 init.cc:377] Logged in from keytab as kudu/127.13.109.62@KRBTEST.COM (short username kudu)
I20260811 19:04:41.355410 5612 webserver.cc:460] Webserver started at http://127.13.109.62:36081/ using document root <none> and password file <none>
I20260811 19:04:41.356014 5612 fs_manager.cc:359] Metadata directory not provided
I20260811 19:04:41.356113 5612 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:04:41.356367 5612 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:04:41.358610 5612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474792496492-13748-0/minicluster-data/master-0/data/instance:
uuid: "a4955674c013467991b3c3d9c425a855"
format_stamp: "Formatted at 2026-08-11 19:04:41 on dist-test-slave-0kls"
server_key: "2baffe61fa1d258b78460d70b104368a"
server_key_iv: "9d8aa044fcc326736c019eee5b179634"
server_key_version: "encryptionkey@0"
I20260811 19:04:41.359215 5612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474792496492-13748-0/minicluster-data/master-0/wal/instance:
uuid: "a4955674c013467991b3c3d9c425a855"
format_stamp: "Formatted at 2026-08-11 19:04:41 on dist-test-slave-0kls"
server_key: "2baffe61fa1d258b78460d70b104368a"
server_key_iv: "9d8aa044fcc326736c019eee5b179634"
server_key_version: "encryptionkey@0"
I20260811 19:04:41.363134 5612 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 19:04:41.365702 5627 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:04:41.366988 5612 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 19:04:41.367141 5612 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474792496492-13748-0/minicluster-data/master-0/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474792496492-13748-0/minicluster-data/master-0/wal
uuid: "a4955674c013467991b3c3d9c425a855"
format_stamp: "Formatted at 2026-08-11 19:04:41 on dist-test-slave-0kls"
server_key: "2baffe61fa1d258b78460d70b104368a"
server_key_iv: "9d8aa044fcc326736c019eee5b179634"
server_key_version: "encryptionkey@0"
I20260811 19:04:41.367264 5612 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474792496492-13748-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474792496492-13748-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474792496492-13748-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:04:41.382333 5612 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:04:41.385448 5612 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:04:41.385679 5612 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:04:41.393280 5612 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.62:43149
I20260811 19:04:41.393414 5683 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.62:43149 every 8 connection(s)
I20260811 19:04:41.394371 5612 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
I20260811 19:04:41.394894 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 5612
I20260811 19:04:41.395037 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474792496492-13748-0/minicluster-data/master-0/wal/instance
I20260811 19:04:41.395316 13748 external_mini_cluster.cc:1428] Setting key 0185d44bd0370fa1526c275a9b2e1ca0
I20260811 19:04:41.397624 5684 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:04:41.404456 5684 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4955674c013467991b3c3d9c425a855: Bootstrap starting.
I20260811 19:04:41.404762 5688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:41.396737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45377 (local address 127.13.109.62:43149)
0811 19:04:41.397163 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:41.397181 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:41.397225 (+ 44us) server_negotiation.cc:407] Connection header received
0811 19:04:41.398025 (+ 800us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:41.398044 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:41.398390 (+ 346us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:41.398667 (+ 277us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:41.399055 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:41.400073 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:41.401297 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:41.401929 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:41.402051 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:41.402965 (+ 914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:04:41.402970 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:04:41.402982 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:04:41.403383 (+ 401us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:04:41.403440 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:04:41.403463 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:04:41.403539 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:41.403724 (+ 185us) server_negotiation.cc:299] Negotiation successful
0811 19:04:41.403963 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":153,"threads_started":1}
I20260811 19:04:41.407091 5684 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a4955674c013467991b3c3d9c425a855: Neither blocks nor log segments found. Creating new log.
I20260811 19:04:41.407800 13748 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 19:04:41.408090 5684 log.cc:824] T 00000000000000000000000000000000 P a4955674c013467991b3c3d9c425a855: Log is configured to *not* fsync() on all Append() calls
I20260811 19:04:41.410405 5684 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4955674c013467991b3c3d9c425a855: No bootstrap required, opened a new log
I20260811 19:04:41.413938 5688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:41.410077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40901 (local address 127.13.109.62:43149)
0811 19:04:41.410221 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:41.410225 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:41.410390 (+ 165us) server_negotiation.cc:407] Connection header received
0811 19:04:41.410525 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:41.410528 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:41.410579 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:41.410660 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:41.410943 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:41.411452 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:41.412253 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:41.412491 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:41.412606 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:41.413443 (+ 837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:04:41.413451 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:04:41.413453 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:04:41.413465 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:04:41.413487 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:04:41.413497 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:04:41.413577 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:41.413701 (+ 124us) server_negotiation.cc:299] Negotiation successful
0811 19:04:41.413751 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 19:04:41.414386 5684 raft_consensus.cc:348] T 00000000000000000000000000000000 P a4955674c013467991b3c3d9c425a855 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4955674c013467991b3c3d9c425a855" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 43149 } }
I20260811 19:04:41.414693 5684 raft_consensus.cc:374] T 00000000000000000000000000000000 P a4955674c013467991b3c3d9c425a855 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:04:41.414758 5684 raft_consensus.cc:729] T 00000000000000000000000000000000 P a4955674c013467991b3c3d9c425a855 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4955674c013467991b3c3d9c425a855, State: Initialized, Role: FOLLOWER
I20260811 19:04:41.415233 5684 consensus_queue.cc:260] T 00000000000000000000000000000000 P a4955674c013467991b3c3d9c425a855 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4955674c013467991b3c3d9c425a855" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 43149 } }
I20260811 19:04:41.415395 5684 raft_consensus.cc:388] T 00000000000000000000000000000000 P a4955674c013467991b3c3d9c425a855 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:04:41.415484 5684 raft_consensus.cc:482] T 00000000000000000000000000000000 P a4955674c013467991b3c3d9c425a855 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:04:41.415616 5684 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a4955674c013467991b3c3d9c425a855 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:04:41.416574 5684 raft_consensus.cc:504] T 00000000000000000000000000000000 P a4955674c013467991b3c3d9c425a855 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4955674c013467991b3c3d9c425a855" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 43149 } }
I20260811 19:04:41.416968 5684 leader_election.cc:302] T 00000000000000000000000000000000 P a4955674c013467991b3c3d9c425a855 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a4955674c013467991b3c3d9c425a855; no voters:
I20260811 19:04:41.417326 5684 leader_election.cc:288] T 00000000000000000000000000000000 P a4955674c013467991b3c3d9c425a855 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:04:41.417440 5697 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a4955674c013467991b3c3d9c425a855 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:04:41.417701 5697 raft_consensus.cc:686] T 00000000000000000000000000000000 P a4955674c013467991b3c3d9c425a855 [term 1 LEADER]: Becoming Leader. State: Replica: a4955674c013467991b3c3d9c425a855, State: Running, Role: LEADER
I20260811 19:04:41.418084 5697 consensus_queue.cc:237] T 00000000000000000000000000000000 P a4955674c013467991b3c3d9c425a855 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4955674c013467991b3c3d9c425a855" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 43149 } }
I20260811 19:04:41.418728 5684 sys_catalog.cc:564] T 00000000000000000000000000000000 P a4955674c013467991b3c3d9c425a855 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:04:41.419790 5699 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4955674c013467991b3c3d9c425a855 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a4955674c013467991b3c3d9c425a855. Latest consensus state: current_term: 1 leader_uuid: "a4955674c013467991b3c3d9c425a855" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4955674c013467991b3c3d9c425a855" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 43149 } } }
I20260811 19:04:41.419939 5699 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4955674c013467991b3c3d9c425a855 [sys.catalog]: This master's current role is: LEADER
I20260811 19:04:41.420332 5698 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4955674c013467991b3c3d9c425a855 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a4955674c013467991b3c3d9c425a855" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4955674c013467991b3c3d9c425a855" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 43149 } } }
I20260811 19:04:41.420442 5698 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4955674c013467991b3c3d9c425a855 [sys.catalog]: This master's current role is: LEADER
I20260811 19:04:41.421370 5707 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:04:41.424736 5707 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:04:41.430933 5707 catalog_manager.cc:1347] Generated new cluster ID: 448c4f86e6e44c2cb2dafb45a86be019
I20260811 19:04:41.431026 5707 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:04:41.444579 5707 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:04:41.445648 5707 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:04:41.450834 5707 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a4955674c013467991b3c3d9c425a855: Generated new TSK 0
I20260811 19:04:41.451733 5707 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 19:04:41.462661 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 5612
2026-08-11T19:04:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1494 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474792496492-13748-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:04:41 dist-test-slave-0kls krb5kdc[5720](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:04:41 dist-test-slave-0kls krb5kdc[5720](info): set up 2 sockets
Aug 11 19:04:41 dist-test-slave-0kls krb5kdc[5720](info): commencing operation
krb5kdc: starting...
W20260811 19:04:42.601234 13748 mini_kdc.cc:121] Time spent starting KDC: real 1.114s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474792496492-13748-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474792496492-13748-0/krb5kdc/test-user.keytab.
Aug 11 19:04:42 dist-test-slave-0kls krb5kdc[5720](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475082, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:04:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:04:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.109.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.109.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.109.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab.
Entry for principal kudu/127.13.109.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab.
I20260811 19:04:42.753281 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474792496492-13748-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474792496492-13748-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474792496492-13748-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.62:39263
--webserver_interface=127.13.109.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab
--principal=kudu/127.13.109.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:39537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.109.62:39263
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:04:42.874378 5737 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:04:42.874693 5737 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:04:42.874737 5737 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:04:42.879040 5737 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:04:42.879142 5737 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:04:42.879163 5737 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:04:42.879181 5737 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:04:42.879197 5737 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:04:42.885286 5737 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:39537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474792496492-13748-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474792496492-13748-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.109.62:39263
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab
--principal=kudu/127.13.109.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.62:39263
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.109.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5737
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474792496492-13748-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:04:42.886708 5737 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:04:42.887884 5737 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:04:42.897542 5743 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:04:42.897548 5745 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:04:42.897634 5737 server_base.cc:1034] running on GCE node
W20260811 19:04:42.897552 5742 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:04:42.898461 5737 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:04:42.899648 5737 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:04:42.900846 5737 hybrid_clock.cc:648] HybridClock initialized: now 1786475082900825 us; error 33 us; skew 500 ppm
Aug 11 19:04:42 dist-test-slave-0kls krb5kdc[5720](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475082, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:04:42.903836 5737 init.cc:377] Logged in from keytab as kudu/127.13.109.62@KRBTEST.COM (short username kudu)
I20260811 19:04:42.904841 5737 webserver.cc:460] Webserver started at http://127.13.109.62:42135/ using document root <none> and password file <none>
I20260811 19:04:42.905397 5737 fs_manager.cc:359] Metadata directory not provided
I20260811 19:04:42.905463 5737 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:04:42.905742 5737 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:04:42.907931 5737 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474792496492-13748-0/minicluster-data/master-0/data/instance:
uuid: "fe4bc09f655d4b3a845f1d83e77b7248"
format_stamp: "Formatted at 2026-08-11 19:04:42 on dist-test-slave-0kls"
server_key: "a7b3d40d62a84345fe8307f9546df6fd"
server_key_iv: "e7b85cf23e66e44937f1bdcd14765bf9"
server_key_version: "encryptionkey@0"
I20260811 19:04:42.908468 5737 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474792496492-13748-0/minicluster-data/master-0/wal/instance:
uuid: "fe4bc09f655d4b3a845f1d83e77b7248"
format_stamp: "Formatted at 2026-08-11 19:04:42 on dist-test-slave-0kls"
server_key: "a7b3d40d62a84345fe8307f9546df6fd"
server_key_iv: "e7b85cf23e66e44937f1bdcd14765bf9"
server_key_version: "encryptionkey@0"
I20260811 19:04:42.912384 5737 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260811 19:04:42.914961 5753 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:04:42.916247 5737 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 19:04:42.916360 5737 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474792496492-13748-0/minicluster-data/master-0/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474792496492-13748-0/minicluster-data/master-0/wal
uuid: "fe4bc09f655d4b3a845f1d83e77b7248"
format_stamp: "Formatted at 2026-08-11 19:04:42 on dist-test-slave-0kls"
server_key: "a7b3d40d62a84345fe8307f9546df6fd"
server_key_iv: "e7b85cf23e66e44937f1bdcd14765bf9"
server_key_version: "encryptionkey@0"
I20260811 19:04:42.916458 5737 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474792496492-13748-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474792496492-13748-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474792496492-13748-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:04:42.921198 5737 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:04:42.924252 5737 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:04:42.924414 5737 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:04:42.932224 5737 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.62:39263
I20260811 19:04:42.932253 5809 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.62:39263 every 8 connection(s)
I20260811 19:04:42.933259 5737 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
I20260811 19:04:42.936254 5810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:04:42.942082 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 5737
I20260811 19:04:42.942162 5810 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe4bc09f655d4b3a845f1d83e77b7248: Bootstrap starting.
I20260811 19:04:42.942235 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474792496492-13748-0/minicluster-data/master-0/wal/instance
I20260811 19:04:42.942487 13748 external_mini_cluster.cc:1428] Setting key 8d99fe274882696fd4a92dd37e47dcd7
I20260811 19:04:42.944929 5810 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fe4bc09f655d4b3a845f1d83e77b7248: Neither blocks nor log segments found. Creating new log.
I20260811 19:04:42.945730 5810 log.cc:824] T 00000000000000000000000000000000 P fe4bc09f655d4b3a845f1d83e77b7248: Log is configured to *not* fsync() on all Append() calls
I20260811 19:04:42.948122 5810 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe4bc09f655d4b3a845f1d83e77b7248: No bootstrap required, opened a new log
I20260811 19:04:42.948169 5813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:42.943982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48659 (local address 127.13.109.62:39263)
0811 19:04:42.944502 (+ 520us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:42.944518 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:42.944556 (+ 38us) server_negotiation.cc:407] Connection header received
0811 19:04:42.945311 (+ 755us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:42.945331 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:42.945642 (+ 311us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:42.946052 (+ 410us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:42.946908 (+ 856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:04:42.946914 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:04:42.946927 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:04:42.946937 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:04:42.946990 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:04:42.947009 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:04:42.947102 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:42.947238 (+ 136us) server_negotiation.cc:299] Negotiation successful
0811 19:04:42.947426 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":150,"threads_started":1}
I20260811 19:04:42.951623 13748 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 19:04:42.952606 5810 raft_consensus.cc:348] T 00000000000000000000000000000000 P fe4bc09f655d4b3a845f1d83e77b7248 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe4bc09f655d4b3a845f1d83e77b7248" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 39263 } }
I20260811 19:04:42.952909 5810 raft_consensus.cc:374] T 00000000000000000000000000000000 P fe4bc09f655d4b3a845f1d83e77b7248 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:04:42.952999 5810 raft_consensus.cc:729] T 00000000000000000000000000000000 P fe4bc09f655d4b3a845f1d83e77b7248 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe4bc09f655d4b3a845f1d83e77b7248, State: Initialized, Role: FOLLOWER
I20260811 19:04:42.953619 5810 consensus_queue.cc:260] T 00000000000000000000000000000000 P fe4bc09f655d4b3a845f1d83e77b7248 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe4bc09f655d4b3a845f1d83e77b7248" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 39263 } }
I20260811 19:04:42.953792 5810 raft_consensus.cc:388] T 00000000000000000000000000000000 P fe4bc09f655d4b3a845f1d83e77b7248 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:04:42.953912 5810 raft_consensus.cc:482] T 00000000000000000000000000000000 P fe4bc09f655d4b3a845f1d83e77b7248 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:04:42.954049 5810 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fe4bc09f655d4b3a845f1d83e77b7248 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:04:42.955482 5810 raft_consensus.cc:504] T 00000000000000000000000000000000 P fe4bc09f655d4b3a845f1d83e77b7248 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe4bc09f655d4b3a845f1d83e77b7248" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 39263 } }
I20260811 19:04:42.955729 5813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:42.953875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52399 (local address 127.13.109.62:39263)
0811 19:04:42.954011 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:42.954016 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:42.954309 (+ 293us) server_negotiation.cc:407] Connection header received
0811 19:04:42.954409 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:42.954412 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:42.954458 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:42.954547 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:42.955327 (+ 780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:04:42.955333 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:04:42.955339 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:04:42.955356 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:04:42.955390 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:04:42.955408 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:04:42.955512 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:42.955569 (+ 57us) server_negotiation.cc:299] Negotiation successful
0811 19:04:42.955613 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 19:04:42.955922 5810 leader_election.cc:302] T 00000000000000000000000000000000 P fe4bc09f655d4b3a845f1d83e77b7248 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fe4bc09f655d4b3a845f1d83e77b7248; no voters:
I20260811 19:04:42.956285 5810 leader_election.cc:288] T 00000000000000000000000000000000 P fe4bc09f655d4b3a845f1d83e77b7248 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:04:42.956393 5820 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fe4bc09f655d4b3a845f1d83e77b7248 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:04:42.956574 5820 raft_consensus.cc:686] T 00000000000000000000000000000000 P fe4bc09f655d4b3a845f1d83e77b7248 [term 1 LEADER]: Becoming Leader. State: Replica: fe4bc09f655d4b3a845f1d83e77b7248, State: Running, Role: LEADER
I20260811 19:04:42.956947 5820 consensus_queue.cc:237] T 00000000000000000000000000000000 P fe4bc09f655d4b3a845f1d83e77b7248 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe4bc09f655d4b3a845f1d83e77b7248" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 39263 } }
I20260811 19:04:42.957939 5810 sys_catalog.cc:564] T 00000000000000000000000000000000 P fe4bc09f655d4b3a845f1d83e77b7248 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:04:42.959095 5821 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe4bc09f655d4b3a845f1d83e77b7248 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fe4bc09f655d4b3a845f1d83e77b7248" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe4bc09f655d4b3a845f1d83e77b7248" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 39263 } } }
I20260811 19:04:42.959260 5821 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe4bc09f655d4b3a845f1d83e77b7248 [sys.catalog]: This master's current role is: LEADER
I20260811 19:04:42.959622 5822 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe4bc09f655d4b3a845f1d83e77b7248 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fe4bc09f655d4b3a845f1d83e77b7248. Latest consensus state: current_term: 1 leader_uuid: "fe4bc09f655d4b3a845f1d83e77b7248" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe4bc09f655d4b3a845f1d83e77b7248" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 39263 } } }
I20260811 19:04:42.959729 5822 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe4bc09f655d4b3a845f1d83e77b7248 [sys.catalog]: This master's current role is: LEADER
I20260811 19:04:42.959721 5829 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:04:42.963425 5829 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:04:42.969071 5829 catalog_manager.cc:1347] Generated new cluster ID: c4e7aec897dd404e88474e18e33d0755
I20260811 19:04:42.969164 5829 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:04:42.978022 5829 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:04:42.978379 5829 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:04:42.988834 5829 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fe4bc09f655d4b3a845f1d83e77b7248: Generated new TSK 0
I20260811 19:04:42.989596 5829 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 19:04:43.002864 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 5737
2026-08-11T19:04:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1539 ms)
[----------] 18 tests from AuthTokenIssuingTest (27145 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474792496492-13748-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:04:43 dist-test-slave-0kls krb5kdc[5846](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:04:43 dist-test-slave-0kls krb5kdc[5846](info): set up 2 sockets
Aug 11 19:04:43 dist-test-slave-0kls krb5kdc[5846](info): commencing operation
krb5kdc: starting...
W20260811 19:04:44.139088 13748 mini_kdc.cc:121] Time spent starting KDC: real 1.112s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474792496492-13748-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474792496492-13748-0/krb5kdc/test-user.keytab.
Aug 11 19:04:44 dist-test-slave-0kls krb5kdc[5846](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475084, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:04:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:04:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.109.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.109.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.109.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab.
Entry for principal kudu/127.13.109.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab.
I20260811 19:04:44.274973 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474792496492-13748-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474792496492-13748-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474792496492-13748-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.62:43309
--webserver_interface=127.13.109.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab
--principal=kudu/127.13.109.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:38587
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.109.62:43309,127.13.109.61:37729,127.13.109.60:44399
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:04:44.396560 5860 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 19:04:44.396996 5860 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:04:44.397081 5860 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:04:44.397174 5860 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:04:44.401428 5860 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:04:44.401576 5860 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:04:44.401623 5860 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:04:44.401669 5860 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:04:44.401724 5860 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:04:44.407464 5860 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:38587
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474792496492-13748-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474792496492-13748-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.109.62:43309,127.13.109.61:37729,127.13.109.60:44399
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab
--principal=kudu/127.13.109.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.62:43309
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.109.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5860
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474792496492-13748-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:04:44.408717 5860 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:04:44.409857 5860 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 19:04:44.419080 5860 server_base.cc:1034] running on GCE node
W20260811 19:04:44.419021 5865 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:04:44.419119 5869 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:04:44.419026 5866 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:04:44.419943 5860 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:04:44.421144 5860 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:04:44.422353 5860 hybrid_clock.cc:648] HybridClock initialized: now 1786475084422335 us; error 35 us; skew 500 ppm
Aug 11 19:04:44 dist-test-slave-0kls krb5kdc[5846](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475084, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:04:44.431365 5860 init.cc:377] Logged in from keytab as kudu/127.13.109.62@KRBTEST.COM (short username kudu)
I20260811 19:04:44.432504 5860 webserver.cc:460] Webserver started at http://127.13.109.62:32895/ using document root <none> and password file <none>
I20260811 19:04:44.433089 5860 fs_manager.cc:359] Metadata directory not provided
I20260811 19:04:44.433154 5860 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:04:44.433336 5860 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:04:44.435568 5860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474792496492-13748-0/minicluster-data/master-0/data/instance:
uuid: "5b78258430ff4796945bf0d4661a8e63"
format_stamp: "Formatted at 2026-08-11 19:04:44 on dist-test-slave-0kls"
server_key: "a47c830946e4fb4c13a1da4d36e012e5"
server_key_iv: "868400f35c5e8591a6d82a021038e2f2"
server_key_version: "encryptionkey@0"
I20260811 19:04:44.436142 5860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474792496492-13748-0/minicluster-data/master-0/wal/instance:
uuid: "5b78258430ff4796945bf0d4661a8e63"
format_stamp: "Formatted at 2026-08-11 19:04:44 on dist-test-slave-0kls"
server_key: "a47c830946e4fb4c13a1da4d36e012e5"
server_key_iv: "868400f35c5e8591a6d82a021038e2f2"
server_key_version: "encryptionkey@0"
I20260811 19:04:44.440239 5860 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260811 19:04:44.442833 5879 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:04:44.444121 5860 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 19:04:44.444235 5860 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474792496492-13748-0/minicluster-data/master-0/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474792496492-13748-0/minicluster-data/master-0/wal
uuid: "5b78258430ff4796945bf0d4661a8e63"
format_stamp: "Formatted at 2026-08-11 19:04:44 on dist-test-slave-0kls"
server_key: "a47c830946e4fb4c13a1da4d36e012e5"
server_key_iv: "868400f35c5e8591a6d82a021038e2f2"
server_key_version: "encryptionkey@0"
I20260811 19:04:44.444332 5860 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474792496492-13748-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474792496492-13748-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474792496492-13748-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:04:44.467788 5860 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:04:44.471354 5860 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:04:44.471529 5860 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:04:44.479557 5860 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.62:43309
I20260811 19:04:44.479885 5937 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.62:43309 every 8 connection(s)
I20260811 19:04:44.480686 5860 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
I20260811 19:04:44.483353 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 5860
I20260811 19:04:44.483466 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474792496492-13748-0/minicluster-data/master-0/wal/instance
I20260811 19:04:44.483775 13748 external_mini_cluster.cc:1428] Setting key 8e56a9236cced166398bf0671cca38cf
I20260811 19:04:44.483834 5938 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:04:44.487938 5938 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.109.62" port: 43309 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.13.109.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.109.61@KRBTEST.COM" created.
Aug 11 19:04:44 dist-test-slave-0kls krb5kdc[5846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475084, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.62@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
I20260811 19:04:44.505699 5941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:44.490727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43238 (local address 127.13.109.62:43309)
0811 19:04:44.491244 (+ 517us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:44.491265 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:44.491567 (+ 302us) server_negotiation.cc:407] Connection header received
0811 19:04:44.492723 (+ 1156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:44.492728 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:44.492996 (+ 268us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:44.493206 (+ 210us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:44.494234 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:44.495216 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:44.496391 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:44.496789 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:44.496984 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:44.499917 (+ 2933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:04:44.499929 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:04:44.499950 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:04:44.499986 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:04:44.502197 (+ 2211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:44.502728 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:44.502737 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:44.502741 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:44.502909 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:44.503258 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:44.503265 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:44.503268 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:44.503961 (+ 693us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:04:44.504126 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:44.504629 (+ 503us) server_negotiation.cc:299] Negotiation successful
0811 19:04:44.504944 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":83,"server-negotiator.queue_time_us":350,"thread_start_us":186,"threads_started":1}
I20260811 19:04:44.505704 5942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:44.490728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:43309 (local address 127.0.0.1:43238)
0811 19:04:44.491404 (+ 676us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:44.491445 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 19:04:44.492378 (+ 933us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:04:44.493200 (+ 822us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:04:44.493206 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:04:44.493669 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:04:44.494054 (+ 385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:44.494071 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:44.495352 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:44.495358 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:44.496264 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:44.496276 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:44.496963 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:44.496967 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:44.497074 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:44.497814 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:04:44.497847 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:04:44.499690 (+ 1843us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:04:44.502331 (+ 2641us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:04:44.502341 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:04:44.502351 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:04:44.502586 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:04:44.503023 (+ 437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:04:44.503026 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:04:44.503027 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:04:44.503156 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:04:44.504189 (+ 1033us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:04:44.504197 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:04:44.504337 (+ 140us) client_negotiation.cc:769] Sending connection context
0811 19:04:44.504493 (+ 156us) client_negotiation.cc:240] Negotiation successful
0811 19:04:44.504802 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":502,"spinlock_wait_cycles":34688,"thread_start_us":183,"threads_started":1}
I20260811 19:04:44.508407 5938 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.109.61" port: 37729 } has no permanent_uuid. Determining permanent_uuid...
I20260811 19:04:44.509297 5942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:44.508970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.61:37729 (local address 127.0.0.1:44082)
0811 19:04:44.509104 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:44.509188 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.61:37729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260811 19:04:44.509861 5880 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.109.61:37729: connect: Connection refused (error 111)
I20260811 19:04:44.511785 5942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:44.511437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.61:37729 (local address 127.0.0.1:44086)
0811 19:04:44.511592 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:44.511691 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.61:37729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260811 19:04:44.511983 5938 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.109.61:37729: Network error: Client connection negotiation failed: client connection to 127.13.109.61:37729: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.109.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.61.keytab.
Entry for principal kudu/127.13.109.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.61.keytab.
I20260811 19:04:44.516425 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474792496492-13748-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474792496492-13748-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474792496492-13748-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474792496492-13748-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.61:37729
--webserver_interface=127.13.109.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.61.keytab
--principal=kudu/127.13.109.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:38587
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.109.62:43309,127.13.109.61:37729,127.13.109.60:44399
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 19:04:44.561337 5938 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.109.61" port: 37729 } attempt: 1
I20260811 19:04:44.562497 5942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:44.562117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.61:37729 (local address 127.0.0.1:44098)
0811 19:04:44.562343 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:44.562413 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.61:37729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119}
I20260811 19:04:44.563541 5942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:44.563267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.61:37729 (local address 127.0.0.1:44102)
0811 19:04:44.563405 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:44.563470 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.61:37729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260811 19:04:44.563712 5938 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.109.61:37729: Network error: Client connection negotiation failed: client connection to 127.13.109.61:37729: connect: Connection refused (error 111)
I20260811 19:04:44.632056 5938 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.109.61" port: 37729 } attempt: 2
I20260811 19:04:44.633005 5942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:44.632683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.61:37729 (local address 127.0.0.1:44110)
0811 19:04:44.632857 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:44.632927 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.61:37729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260811 19:04:44.633931 5942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:44.633617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.61:37729 (local address 127.0.0.1:44118)
0811 19:04:44.633754 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:44.633861 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.61:37729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260811 19:04:44.634089 5938 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.109.61:37729: Network error: Client connection negotiation failed: client connection to 127.13.109.61:37729: connect: Connection refused (error 111)
W20260811 19:04:44.637400 5946 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 19:04:44.637713 5946 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:04:44.637827 5946 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:04:44.637898 5946 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:04:44.642086 5946 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:04:44.642151 5946 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:04:44.642170 5946 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:04:44.642187 5946 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:04:44.642251 5946 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:04:44.647969 5946 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:38587
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474792496492-13748-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474792496492-13748-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.13.109.62:43309,127.13.109.61:37729,127.13.109.60:44399
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.61.keytab
--principal=kudu/127.13.109.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.61:37729
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474792496492-13748-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.13.109.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5946
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474792496492-13748-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:04:44.649255 5946 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:04:44.650427 5946 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:04:44.659569 5954 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:04:44.659591 5956 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:04:44.659570 5953 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:04:44.659835 5946 server_base.cc:1034] running on GCE node
I20260811 19:04:44.660507 5946 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:04:44.661676 5946 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:04:44.662892 5946 hybrid_clock.cc:648] HybridClock initialized: now 1786475084662861 us; error 46 us; skew 500 ppm
Aug 11 19:04:44 dist-test-slave-0kls krb5kdc[5846](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475084, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:04:44.666013 5946 init.cc:377] Logged in from keytab as kudu/127.13.109.61@KRBTEST.COM (short username kudu)
I20260811 19:04:44.667018 5946 webserver.cc:460] Webserver started at http://127.13.109.61:43199/ using document root <none> and password file <none>
I20260811 19:04:44.667616 5946 fs_manager.cc:359] Metadata directory not provided
I20260811 19:04:44.667713 5946 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:04:44.667968 5946 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:04:44.670210 5946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474792496492-13748-0/minicluster-data/master-1/data/instance:
uuid: "948dc57da5194edab5a24f9948b31fd7"
format_stamp: "Formatted at 2026-08-11 19:04:44 on dist-test-slave-0kls"
server_key: "9b01f4d73fba0e96080eebddcd22f5ba"
server_key_iv: "e7127620dae1ff99d98025ea3657abcc"
server_key_version: "encryptionkey@0"
I20260811 19:04:44.670804 5946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474792496492-13748-0/minicluster-data/master-1/wal/instance:
uuid: "948dc57da5194edab5a24f9948b31fd7"
format_stamp: "Formatted at 2026-08-11 19:04:44 on dist-test-slave-0kls"
server_key: "9b01f4d73fba0e96080eebddcd22f5ba"
server_key_iv: "e7127620dae1ff99d98025ea3657abcc"
server_key_version: "encryptionkey@0"
I20260811 19:04:44.674787 5946 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 19:04:44.677320 5963 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:04:44.678642 5946 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 19:04:44.678800 5946 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474792496492-13748-0/minicluster-data/master-1/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474792496492-13748-0/minicluster-data/master-1/wal
uuid: "948dc57da5194edab5a24f9948b31fd7"
format_stamp: "Formatted at 2026-08-11 19:04:44 on dist-test-slave-0kls"
server_key: "9b01f4d73fba0e96080eebddcd22f5ba"
server_key_iv: "e7127620dae1ff99d98025ea3657abcc"
server_key_version: "encryptionkey@0"
I20260811 19:04:44.678925 5946 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474792496492-13748-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474792496492-13748-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474792496492-13748-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:04:44.689410 5946 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:04:44.692845 5946 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:04:44.693073 5946 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:04:44.700726 5946 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.61:37729
I20260811 19:04:44.701040 6018 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.61:37729 every 8 connection(s)
I20260811 19:04:44.701844 5946 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474792496492-13748-0/minicluster-data/master-1/data/info.pb
I20260811 19:04:44.704499 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 5946
I20260811 19:04:44.704631 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474792496492-13748-0/minicluster-data/master-1/wal/instance
I20260811 19:04:44.704974 13748 external_mini_cluster.cc:1428] Setting key b12bdefd159024bc2224c1f7e708df90
I20260811 19:04:44.704973 6020 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:04:44.709198 6020 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.109.62" port: 43309 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.13.109.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.109.60@KRBTEST.COM" created.
Aug 11 19:04:44 dist-test-slave-0kls krb5kdc[5846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475084, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.61@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
I20260811 19:04:44.724865 5941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:44.711360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43254 (local address 127.13.109.62:43309)
0811 19:04:44.711570 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:44.711578 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:44.712495 (+ 917us) server_negotiation.cc:407] Connection header received
0811 19:04:44.713487 (+ 992us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:44.713493 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:44.713561 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:44.713676 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:44.715064 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:44.715613 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:44.716872 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:44.717266 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:44.717399 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:44.720375 (+ 2976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:04:44.720381 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:04:44.720386 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:04:44.720425 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:04:44.722166 (+ 1741us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:44.722886 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:44.722889 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:44.722891 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:44.722987 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:44.723457 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:44.723460 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:44.723461 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:44.723668 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:04:44.723748 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:44.724575 (+ 827us) server_negotiation.cc:299] Negotiation successful
0811 19:04:44.724722 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 19:04:44.725271 5938 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.109.61" port: 37729 } attempt: 3
I20260811 19:04:44.726274 6023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:44.711721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:43309 (local address 127.0.0.1:43254)
0811 19:04:44.712304 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:44.712359 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 19:04:44.713254 (+ 895us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:04:44.713906 (+ 652us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:04:44.713913 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:04:44.714326 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:04:44.714840 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:44.714855 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:44.715750 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:44.715752 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:44.716745 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:44.716756 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:44.717457 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:44.717460 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:44.717666 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:44.718354 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:04:44.718383 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:04:44.720156 (+ 1773us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:04:44.722306 (+ 2150us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:04:44.722316 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:04:44.722338 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:04:44.722748 (+ 410us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:04:44.723107 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:04:44.723113 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:04:44.723118 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:04:44.723347 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:04:44.723794 (+ 447us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:04:44.723802 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:04:44.724233 (+ 431us) client_negotiation.cc:769] Sending connection context
0811 19:04:44.724497 (+ 264us) client_negotiation.cc:240] Negotiation successful
0811 19:04:44.724936 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":172,"threads_started":1}
I20260811 19:04:44.727815 6020 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.109.61" port: 37729 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 19:04:44 dist-test-slave-0kls krb5kdc[5846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475084, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.62@KRBTEST.COM for kudu/127.13.109.61@KRBTEST.COM
Aug 11 19:04:44 dist-test-slave-0kls krb5kdc[5846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475084, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.61@KRBTEST.COM for kudu/127.13.109.61@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.109.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.60.keytab.
Entry for principal kudu/127.13.109.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.60.keytab.
I20260811 19:04:44.737285 5942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:44.725771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.61:37729 (local address 127.0.0.1:44132)
0811 19:04:44.726513 (+ 742us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:44.726529 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 19:04:44.726616 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:04:44.727442 (+ 826us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:04:44.727445 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:04:44.727647 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:04:44.727792 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:44.727801 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:44.729043 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:44.729049 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:44.729701 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:44.729711 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:44.730386 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:44.730390 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:44.730507 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:44.731086 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:04:44.731101 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:04:44.732649 (+ 1548us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:04:44.735248 (+ 2599us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:04:44.735251 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:04:44.735252 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:04:44.735562 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:04:44.735914 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:04:44.735917 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:04:44.735918 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:04:44.736036 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:04:44.736702 (+ 666us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:04:44.736705 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:04:44.736813 (+ 108us) client_negotiation.cc:769] Sending connection context
0811 19:04:44.736929 (+ 116us) client_negotiation.cc:240] Negotiation successful
0811 19:04:44.737092 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":650}
I20260811 19:04:44.737418 6027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:44.726044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44132 (local address 127.13.109.61:37729)
0811 19:04:44.726946 (+ 902us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:44.726959 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:44.726982 (+ 23us) server_negotiation.cc:407] Connection header received
0811 19:04:44.727027 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:44.727033 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:44.727285 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:44.727426 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:44.727908 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:44.728919 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:44.729854 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:44.730243 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:44.730371 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:44.732827 (+ 2456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:04:44.732838 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:04:44.732852 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:04:44.732874 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:04:44.735128 (+ 2254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:44.735683 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:44.735691 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:44.735693 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:44.735788 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:44.736141 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:44.736147 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:44.736152 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:44.736512 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:04:44.736671 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:44.737043 (+ 372us) server_negotiation.cc:299] Negotiation successful
0811 19:04:44.737225 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":786,"thread_start_us":202,"threads_started":1}
I20260811 19:04:44.737488 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474792496492-13748-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474792496492-13748-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474792496492-13748-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474792496492-13748-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.60:44399
--webserver_interface=127.13.109.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.60.keytab
--principal=kudu/127.13.109.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:38587
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.109.62:43309,127.13.109.61:37729,127.13.109.60:44399
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 19:04:44.738680 6023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:44.728400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.61:37729 (local address 127.0.0.1:44142)
0811 19:04:44.728547 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:44.728561 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 19:04:44.728670 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:04:44.728977 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:04:44.728980 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:04:44.729159 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:04:44.729237 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:44.729242 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:44.730130 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:44.730135 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:44.730800 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:44.730812 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:44.731517 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:44.731520 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:44.731593 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:44.732120 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:04:44.732134 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:04:44.733567 (+ 1433us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:04:44.736325 (+ 2758us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:04:44.736328 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:04:44.736329 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:04:44.736633 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:04:44.737598 (+ 965us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:04:44.737601 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:04:44.737603 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:04:44.737701 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:04:44.738204 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:04:44.738209 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:04:44.738307 (+ 98us) client_negotiation.cc:769] Sending connection context
0811 19:04:44.738411 (+ 104us) client_negotiation.cc:240] Negotiation successful
0811 19:04:44.738537 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73}
I20260811 19:04:44.738921 6028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:44.728440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44142 (local address 127.13.109.61:37729)
0811 19:04:44.728787 (+ 347us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:44.728799 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:44.728808 (+ 9us) server_negotiation.cc:407] Connection header received
0811 19:04:44.728832 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:44.728834 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:44.728864 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:44.728966 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:44.729345 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:44.730006 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:44.730962 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:44.731357 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:44.731463 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:44.733733 (+ 2270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:04:44.733742 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:04:44.733744 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:04:44.733767 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:04:44.736220 (+ 2453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:44.736746 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:44.736752 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:44.736757 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:44.736839 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:44.737842 (+ 1003us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:44.737848 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:44.737852 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:44.738095 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:04:44.738208 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:44.738484 (+ 276us) server_negotiation.cc:299] Negotiation successful
0811 19:04:44.738622 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":122,"threads_started":1}
I20260811 19:04:44.740414 5938 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.109.60" port: 44399 } has no permanent_uuid. Determining permanent_uuid...
I20260811 19:04:44.740592 6020 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.109.60" port: 44399 } has no permanent_uuid. Determining permanent_uuid...
I20260811 19:04:44.741245 5942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:44.740951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.60:44399 (local address 127.0.0.1:57196)
0811 19:04:44.741106 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:44.741165 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.60:44399: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260811 19:04:44.741384 6023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:44.741073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.60:44399 (local address 127.0.0.1:57202)
0811 19:04:44.741207 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:44.741296 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.60:44399: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260811 19:04:44.741832 5967 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.109.60:44399: connect: Connection refused (error 111)
I20260811 19:04:44.742121 5942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:44.741775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.60:44399 (local address 127.0.0.1:57204)
0811 19:04:44.741976 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:44.742038 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.60:44399: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
W20260811 19:04:44.742307 5938 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.109.60:44399: Network error: Client connection negotiation failed: client connection to 127.13.109.60:44399: connect: Connection refused (error 111)
I20260811 19:04:44.742951 6023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:44.742658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.60:44399 (local address 127.0.0.1:57208)
0811 19:04:44.742807 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:44.742869 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.60:44399: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260811 19:04:44.743201 6020 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.109.60:44399: Network error: Client connection negotiation failed: client connection to 127.13.109.60:44399: connect: Connection refused (error 111)
I20260811 19:04:44.773499 5938 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.109.60" port: 44399 } attempt: 1
I20260811 19:04:44.774506 5942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:44.774127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.60:44399 (local address 127.0.0.1:57210)
0811 19:04:44.774330 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:44.774408 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.60:44399: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260811 19:04:44.775311 5942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:44.775062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.60:44399 (local address 127.0.0.1:57220)
0811 19:04:44.775182 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:44.775249 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.60:44399: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260811 19:04:44.775465 5938 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.109.60:44399: Network error: Client connection negotiation failed: client connection to 127.13.109.60:44399: connect: Connection refused (error 111)
I20260811 19:04:44.792520 6020 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.109.60" port: 44399 } attempt: 1
I20260811 19:04:44.793555 6023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:44.793190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.60:44399 (local address 127.0.0.1:57236)
0811 19:04:44.793373 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:44.793451 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.60:44399: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260811 19:04:44.794478 6023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:44.794206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.60:44399 (local address 127.0.0.1:57250)
0811 19:04:44.794338 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:44.794406 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.60:44399: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260811 19:04:44.794672 6020 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.109.60:44399: Network error: Client connection negotiation failed: client connection to 127.13.109.60:44399: connect: Connection refused (error 111)
I20260811 19:04:44.850673 5938 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.109.60" port: 44399 } attempt: 2
I20260811 19:04:44.851570 5942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:44.851264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.60:44399 (local address 127.0.0.1:57266)
0811 19:04:44.851426 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:44.851491 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.60:44399: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260811 19:04:44.852305 5942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:44.852086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.60:44399 (local address 127.0.0.1:57274)
0811 19:04:44.852200 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:44.852253 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.60:44399: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 19:04:44.852424 5938 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.109.60:44399: Network error: Client connection negotiation failed: client connection to 127.13.109.60:44399: connect: Connection refused (error 111)
W20260811 19:04:44.859444 6029 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 19:04:44.859813 6029 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:04:44.859879 6029 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:04:44.859961 6029 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260811 19:04:44.862854 6020 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.109.60" port: 44399 } attempt: 2
I20260811 19:04:44.863750 6023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:44.863397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.60:44399 (local address 127.0.0.1:57290)
0811 19:04:44.863580 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:44.863657 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.60:44399: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260811 19:04:44.864545 6023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:44.864331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.60:44399 (local address 127.0.0.1:57306)
0811 19:04:44.864436 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:44.864487 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.60:44399: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260811 19:04:44.864566 6029 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:04:44.864619 6029 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:04:44.864652 6029 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:04:44.864681 6029 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:04:44.864725 6029 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 19:04:44.864710 6020 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.109.60:44399: Network error: Client connection negotiation failed: client connection to 127.13.109.60:44399: connect: Connection refused (error 111)
I20260811 19:04:44.870441 6029 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:38587
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474792496492-13748-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474792496492-13748-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.13.109.62:43309,127.13.109.61:37729,127.13.109.60:44399
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.60.keytab
--principal=kudu/127.13.109.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.60:44399
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474792496492-13748-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.13.109.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6029
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474792496492-13748-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:04:44.871596 6029 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:04:44.872644 6029 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:04:44.881450 6035 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:04:44.881425 6036 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:04:44.881474 6039 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:04:44.881750 6029 server_base.cc:1034] running on GCE node
I20260811 19:04:44.882450 6029 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:04:44.883721 6029 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:04:44.884917 6029 hybrid_clock.cc:648] HybridClock initialized: now 1786475084884897 us; error 34 us; skew 500 ppm
Aug 11 19:04:44 dist-test-slave-0kls krb5kdc[5846](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475084, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:04:44.887601 6029 init.cc:377] Logged in from keytab as kudu/127.13.109.60@KRBTEST.COM (short username kudu)
I20260811 19:04:44.888477 6029 webserver.cc:460] Webserver started at http://127.13.109.60:34601/ using document root <none> and password file <none>
I20260811 19:04:44.889045 6029 fs_manager.cc:359] Metadata directory not provided
I20260811 19:04:44.889166 6029 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:04:44.889410 6029 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:04:44.891605 6029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474792496492-13748-0/minicluster-data/master-2/data/instance:
uuid: "5b1cb60521c74879877565e7d56818ed"
format_stamp: "Formatted at 2026-08-11 19:04:44 on dist-test-slave-0kls"
server_key: "23b8a10f2d961f336816583256bb3503"
server_key_iv: "d70039c8efb135760a07721c21b4f4f8"
server_key_version: "encryptionkey@0"
I20260811 19:04:44.892212 6029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474792496492-13748-0/minicluster-data/master-2/wal/instance:
uuid: "5b1cb60521c74879877565e7d56818ed"
format_stamp: "Formatted at 2026-08-11 19:04:44 on dist-test-slave-0kls"
server_key: "23b8a10f2d961f336816583256bb3503"
server_key_iv: "d70039c8efb135760a07721c21b4f4f8"
server_key_version: "encryptionkey@0"
I20260811 19:04:44.896028 6029 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 19:04:44.898957 6048 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:04:44.900293 6029 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 19:04:44.900470 6029 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474792496492-13748-0/minicluster-data/master-2/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474792496492-13748-0/minicluster-data/master-2/wal
uuid: "5b1cb60521c74879877565e7d56818ed"
format_stamp: "Formatted at 2026-08-11 19:04:44 on dist-test-slave-0kls"
server_key: "23b8a10f2d961f336816583256bb3503"
server_key_iv: "d70039c8efb135760a07721c21b4f4f8"
server_key_version: "encryptionkey@0"
I20260811 19:04:44.900600 6029 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474792496492-13748-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474792496492-13748-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474792496492-13748-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:04:44.916142 6029 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:04:44.919153 6029 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:04:44.919385 6029 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:04:44.926736 6029 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.60:44399
I20260811 19:04:44.927013 6107 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.60:44399 every 8 connection(s)
I20260811 19:04:44.927829 6029 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474792496492-13748-0/minicluster-data/master-2/data/info.pb
I20260811 19:04:44.930800 6108 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:04:44.934831 6108 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.109.62" port: 43309 } has no permanent_uuid. Determining permanent_uuid...
I20260811 19:04:44.936666 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 6029
I20260811 19:04:44.936805 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474792496492-13748-0/minicluster-data/master-2/wal/instance
I20260811 19:04:44.937100 13748 external_mini_cluster.cc:1428] Setting key 09928b2507bc3519423c72187c911f29
Aug 11 19:04:44 dist-test-slave-0kls krb5kdc[5846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475084, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.60@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
Aug 11 19:04:44 dist-test-slave-0kls krb5kdc[5846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475084, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.109.60@KRBTEST.COM
I20260811 19:04:44.949271 5941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:44.936961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43260 (local address 127.13.109.62:43309)
0811 19:04:44.937102 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:44.937105 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:44.938299 (+ 1194us) server_negotiation.cc:407] Connection header received
0811 19:04:44.939313 (+ 1014us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:44.939316 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:44.939382 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:44.939474 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:44.940336 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:44.940854 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:44.941910 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:44.942119 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:44.942239 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:44.945152 (+ 2913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:04:44.945159 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:04:44.945166 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:04:44.945204 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:04:44.946801 (+ 1597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:44.947393 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:44.947396 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:44.947398 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:44.947475 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:44.947867 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:44.947871 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:44.947873 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:44.948167 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:04:44.948305 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:44.948944 (+ 639us) server_negotiation.cc:299] Negotiation successful
0811 19:04:44.949084 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 19:04:44.950270 6110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:44.937480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:43309 (local address 127.0.0.1:43260)
0811 19:04:44.938112 (+ 632us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:44.938175 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0811 19:04:44.939100 (+ 925us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:04:44.939529 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:04:44.939534 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:04:44.939902 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:04:44.940183 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:44.940204 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:44.940964 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:44.940969 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:44.941720 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:44.941729 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:44.942300 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:44.942305 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:44.942485 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:44.943097 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:04:44.943121 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:04:44.944898 (+ 1777us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:04:44.946920 (+ 2022us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:04:44.946931 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:04:44.946946 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:04:44.947292 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:04:44.947569 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:04:44.947573 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:04:44.947574 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:04:44.947769 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:04:44.948305 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:04:44.948314 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:04:44.948615 (+ 301us) client_negotiation.cc:769] Sending connection context
0811 19:04:44.948881 (+ 266us) client_negotiation.cc:240] Negotiation successful
0811 19:04:44.949243 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":389,"mutex_wait_us":3,"thread_start_us":182,"threads_started":1}
I20260811 19:04:44.950273 6113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:44.937809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57314 (local address 127.13.109.60:44399)
0811 19:04:44.938273 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:44.938285 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:44.938327 (+ 42us) server_negotiation.cc:407] Connection header received
0811 19:04:44.939080 (+ 753us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:44.939098 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:44.939294 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:44.939495 (+ 201us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:44.940367 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:44.941223 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:44.942191 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:44.942640 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:44.942763 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:44.945545 (+ 2782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:04:44.945553 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:04:44.945570 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:04:44.945601 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:04:44.947582 (+ 1981us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:44.948126 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:44.948132 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:44.948133 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:44.948245 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:44.948526 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:44.948530 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:44.948532 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:44.948792 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:04:44.949002 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:44.949233 (+ 231us) server_negotiation.cc:299] Negotiation successful
0811 19:04:44.949390 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":29,"server-negotiator.queue_time_us":373,"thread_start_us":153,"threads_started":1}
I20260811 19:04:44.951586 5938 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.109.60" port: 44399 } attempt: 3
I20260811 19:04:44.951700 6108 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.109.61" port: 37729 } has no permanent_uuid. Determining permanent_uuid...
I20260811 19:04:44.955880 6020 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.109.60" port: 44399 } attempt: 3
Aug 11 19:04:44 dist-test-slave-0kls krb5kdc[5846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475084, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.62@KRBTEST.COM for kudu/127.13.109.60@KRBTEST.COM
Aug 11 19:04:44 dist-test-slave-0kls krb5kdc[5846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475084, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.60@KRBTEST.COM for kudu/127.13.109.61@KRBTEST.COM
Aug 11 19:04:44 dist-test-slave-0kls krb5kdc[5846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475084, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
I20260811 19:04:44.962765 5942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:44.952058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.60:44399 (local address 127.0.0.1:57320)
0811 19:04:44.952177 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:44.952195 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 19:04:44.952288 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:04:44.952738 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:04:44.952741 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:04:44.952954 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:04:44.953079 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:44.953087 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:44.954318 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:44.954322 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:44.955188 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:44.955200 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:44.955659 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:44.955663 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:44.955751 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:44.956365 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:04:44.956386 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:04:44.958138 (+ 1752us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:04:44.960935 (+ 2797us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:04:44.960939 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:04:44.960941 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:04:44.961211 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:04:44.961541 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:04:44.961545 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:04:44.961548 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:04:44.961647 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:04:44.962161 (+ 514us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:04:44.962165 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:04:44.962289 (+ 124us) client_negotiation.cc:769] Sending connection context
0811 19:04:44.962417 (+ 128us) client_negotiation.cc:240] Negotiation successful
0811 19:04:44.962590 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34,"mutex_wait_us":22}
I20260811 19:04:44.962765 6113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:44.952389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57320 (local address 127.13.109.60:44399)
0811 19:04:44.952489 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:44.952493 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:44.952505 (+ 12us) server_negotiation.cc:407] Connection header received
0811 19:04:44.952541 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:44.952544 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:44.952594 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:44.953206 (+ 612us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:44.953309 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:44.954138 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:44.955280 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:44.955536 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:44.955651 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:44.958318 (+ 2667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:04:44.958322 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:04:44.958325 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:04:44.958358 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:04:44.960811 (+ 2453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:44.961326 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:44.961330 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:44.961332 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:44.961424 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:44.961760 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:44.961764 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:44.961768 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:44.962043 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:04:44.962154 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:44.962471 (+ 317us) server_negotiation.cc:299] Negotiation successful
0811 19:04:44.962584 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260811 19:04:44.962881 6028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:44.952527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44156 (local address 127.13.109.61:37729)
0811 19:04:44.952656 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:44.952663 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:44.952680 (+ 17us) server_negotiation.cc:407] Connection header received
0811 19:04:44.952723 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:44.952726 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:44.952774 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:44.952865 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:44.953345 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:44.954086 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:44.955213 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:44.955587 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:44.955691 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:44.959014 (+ 3323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:04:44.959018 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:04:44.959021 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:04:44.959052 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:04:44.960854 (+ 1802us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:44.961271 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:44.961276 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:44.961278 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:44.961366 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:44.961931 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:44.961934 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:44.961935 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:44.962161 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:04:44.962265 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:44.962557 (+ 292us) server_negotiation.cc:299] Negotiation successful
0811 19:04:44.962697 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 19:04:44.963373 6110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:44.952201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.61:37729 (local address 127.0.0.1:44156)
0811 19:04:44.952531 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:44.952545 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 19:04:44.952618 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:04:44.952879 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:04:44.952882 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:04:44.953108 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:04:44.953236 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:44.953243 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:44.954198 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:44.954203 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:44.955071 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:44.955082 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:44.955828 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:44.955831 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:44.955955 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:44.957123 (+ 1168us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:04:44.957140 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:04:44.958837 (+ 1697us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:04:44.960970 (+ 2133us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:04:44.960973 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:04:44.960981 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:04:44.961165 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:04:44.961466 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:04:44.961470 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:04:44.961471 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:04:44.961564 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:04:44.962251 (+ 687us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:04:44.962257 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:04:44.962354 (+ 97us) client_negotiation.cc:769] Sending connection context
0811 19:04:44.963041 (+ 687us) client_negotiation.cc:240] Negotiation successful
0811 19:04:44.963181 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258}
I20260811 19:04:44.964857 6108 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.109.60" port: 44399 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 19:04:44 dist-test-slave-0kls krb5kdc[5846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475084, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.61@KRBTEST.COM for kudu/127.13.109.60@KRBTEST.COM
I20260811 19:04:44.965305 5941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:44.953052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43266 (local address 127.13.109.62:43309)
0811 19:04:44.953184 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:44.953190 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:44.953256 (+ 66us) server_negotiation.cc:407] Connection header received
0811 19:04:44.953418 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:44.953421 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:44.953464 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:44.953544 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:44.954373 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:44.954873 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:44.955968 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:44.956782 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:44.956863 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:44.960096 (+ 3233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:04:44.960100 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:04:44.960103 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:04:44.960130 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:04:44.962544 (+ 2414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:44.963046 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:44.963051 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:44.963054 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:44.963160 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:44.963487 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:44.963490 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:44.963492 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:44.963731 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:04:44.964555 (+ 824us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:44.964977 (+ 422us) server_negotiation.cc:299] Negotiation successful
0811 19:04:44.965115 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":55}
I20260811 19:04:44.967917 5938 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b78258430ff4796945bf0d4661a8e63: Bootstrap starting.
I20260811 19:04:44.968992 6023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:44.956366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.60:44399 (local address 127.0.0.1:57334)
0811 19:04:44.956585 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:44.956603 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 19:04:44.956688 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:04:44.958464 (+ 1776us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:04:44.958470 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:04:44.958663 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:04:44.958802 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:44.958813 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:44.960469 (+ 1656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:44.960473 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:44.961166 (+ 693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:44.961178 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:44.963385 (+ 2207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:44.963392 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:44.963497 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:44.963927 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:04:44.963940 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:04:44.965316 (+ 1376us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:04:44.967412 (+ 2096us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:04:44.967415 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:04:44.967417 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:04:44.967679 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:04:44.967954 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:04:44.967957 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:04:44.967959 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:04:44.968080 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:04:44.968454 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:04:44.968458 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:04:44.968574 (+ 116us) client_negotiation.cc:769] Sending connection context
0811 19:04:44.968671 (+ 97us) client_negotiation.cc:240] Negotiation successful
0811 19:04:44.968805 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119}
I20260811 19:04:44.969878 6114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:44.956410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57334 (local address 127.13.109.60:44399)
0811 19:04:44.956910 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:44.956916 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:44.956926 (+ 10us) server_negotiation.cc:407] Connection header received
0811 19:04:44.956953 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:44.956955 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:44.956988 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:44.958431 (+ 1443us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:44.958955 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:44.960322 (+ 1367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:44.961649 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:44.963248 (+ 1599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:44.963730 (+ 482us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:44.965479 (+ 1749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:04:44.965482 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:04:44.965488 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:04:44.965517 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:04:44.967294 (+ 1777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:44.967764 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:44.967767 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:44.967768 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:44.967854 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:44.968165 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:44.968169 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:44.968171 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:44.968346 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:04:44.968450 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:44.969568 (+ 1118us) server_negotiation.cc:299] Negotiation successful
0811 19:04:44.969668 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":439,"thread_start_us":115,"threads_started":1}
Aug 11 19:04:44 dist-test-slave-0kls krb5kdc[5846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475084, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.60@KRBTEST.COM for kudu/127.13.109.60@KRBTEST.COM
I20260811 19:04:44.971822 5938 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5b78258430ff4796945bf0d4661a8e63: Neither blocks nor log segments found. Creating new log.
I20260811 19:04:44.972918 5938 log.cc:824] T 00000000000000000000000000000000 P 5b78258430ff4796945bf0d4661a8e63: Log is configured to *not* fsync() on all Append() calls
I20260811 19:04:44.973342 6020 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 948dc57da5194edab5a24f9948b31fd7: Bootstrap starting.
I20260811 19:04:44.974592 6113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:44.965756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57338 (local address 127.13.109.60:44399)
0811 19:04:44.965880 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:44.965884 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:44.966158 (+ 274us) server_negotiation.cc:407] Connection header received
0811 19:04:44.966290 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:44.966294 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:44.966330 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:44.966433 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:44.967006 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:44.967598 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:44.968539 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:44.968848 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:44.968945 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:44.971126 (+ 2181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:04:44.971129 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:04:44.971131 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:04:44.971152 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:04:44.972825 (+ 1673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:44.973290 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:44.973295 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:44.973300 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:44.973384 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:44.973656 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:44.973662 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:44.973666 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:44.973931 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:04:44.974012 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:44.974327 (+ 315us) server_negotiation.cc:299] Negotiation successful
0811 19:04:44.974439 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 19:04:44.975068 6110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:44.965346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.60:44399 (local address 127.0.0.1:57338)
0811 19:04:44.966078 (+ 732us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:44.966089 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 19:04:44.966197 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:04:44.966663 (+ 466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:04:44.966665 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:04:44.966825 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:04:44.966893 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:44.966898 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:44.967722 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:44.967725 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:44.968431 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:44.968441 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:44.968967 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:44.968970 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:44.969047 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:44.969454 (+ 407us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:04:44.969468 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:04:44.970995 (+ 1527us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:04:44.972940 (+ 1945us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:04:44.972942 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:04:44.972944 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:04:44.973171 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:04:44.973461 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:04:44.973465 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:04:44.973468 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:04:44.973555 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:04:44.974079 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:04:44.974085 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:04:44.974169 (+ 84us) client_negotiation.cc:769] Sending connection context
0811 19:04:44.974803 (+ 634us) client_negotiation.cc:240] Negotiation successful
0811 19:04:44.974937 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":684}
I20260811 19:04:44.975618 5938 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b78258430ff4796945bf0d4661a8e63: No bootstrap required, opened a new log
I20260811 19:04:44.976347 6020 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 948dc57da5194edab5a24f9948b31fd7: Neither blocks nor log segments found. Creating new log.
I20260811 19:04:44.977387 6020 log.cc:824] T 00000000000000000000000000000000 P 948dc57da5194edab5a24f9948b31fd7: Log is configured to *not* fsync() on all Append() calls
I20260811 19:04:44.978168 6108 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b1cb60521c74879877565e7d56818ed: Bootstrap starting.
Aug 11 19:04:44 dist-test-slave-0kls krb5kdc[5846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475084, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.109.61@KRBTEST.COM
I20260811 19:04:44.979887 5938 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5b78258430ff4796945bf0d4661a8e63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b78258430ff4796945bf0d4661a8e63" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 43309 } } peers { permanent_uuid: "948dc57da5194edab5a24f9948b31fd7" member_type: VOTER last_known_addr { host: "127.13.109.61" port: 37729 } } peers { permanent_uuid: "5b1cb60521c74879877565e7d56818ed" member_type: VOTER last_known_addr { host: "127.13.109.60" port: 44399 } }
I20260811 19:04:44.980134 5938 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5b78258430ff4796945bf0d4661a8e63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b78258430ff4796945bf0d4661a8e63, State: Initialized, Role: FOLLOWER
I20260811 19:04:44.980486 5938 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5b78258430ff4796945bf0d4661a8e63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b78258430ff4796945bf0d4661a8e63" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 43309 } } peers { permanent_uuid: "948dc57da5194edab5a24f9948b31fd7" member_type: VOTER last_known_addr { host: "127.13.109.61" port: 37729 } } peers { permanent_uuid: "5b1cb60521c74879877565e7d56818ed" member_type: VOTER last_known_addr { host: "127.13.109.60" port: 44399 } }
I20260811 19:04:44.980998 6108 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5b1cb60521c74879877565e7d56818ed: Neither blocks nor log segments found. Creating new log.
I20260811 19:04:44.981737 6020 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 948dc57da5194edab5a24f9948b31fd7: No bootstrap required, opened a new log
I20260811 19:04:44.981910 6108 log.cc:824] T 00000000000000000000000000000000 P 5b1cb60521c74879877565e7d56818ed: Log is configured to *not* fsync() on all Append() calls
I20260811 19:04:44.981989 5938 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5b78258430ff4796945bf0d4661a8e63 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:04:44.983258 6028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:44.971386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44170 (local address 127.13.109.61:37729)
0811 19:04:44.971494 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:44.971499 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:44.971661 (+ 162us) server_negotiation.cc:407] Connection header received
0811 19:04:44.971835 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:44.971840 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:44.971892 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:44.971946 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:44.972895 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:44.973628 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:44.975409 (+ 1781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:44.975701 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:44.975781 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:44.979003 (+ 3222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:04:44.979008 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:04:44.979010 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:04:44.979041 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:04:44.980906 (+ 1865us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:44.981398 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:44.981403 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:44.981404 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:44.981497 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:44.982058 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:44.982062 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:44.982064 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:44.982313 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:04:44.982479 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:44.982934 (+ 455us) server_negotiation.cc:299] Negotiation successful
0811 19:04:44.983090 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 19:04:44.986263 6108 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b1cb60521c74879877565e7d56818ed: No bootstrap required, opened a new log
I20260811 19:04:44.987123 13748 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260811 19:04:44.987339 6129 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5b78258430ff4796945bf0d4661a8e63: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 19:04:44.987408 6129 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 19:04:44.988925 5910 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "5b78258430ff4796945bf0d4661a8e63"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43266
I20260811 19:04:44.989269 5910 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5b78258430ff4796945bf0d4661a8e63 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260811 19:04:44.989388 5910 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5b78258430ff4796945bf0d4661a8e63 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:04:44.987077 5860 master_runner.cc:186] Error getting master registration for 127.13.109.61:37729: OK, instance_id { permanent_uuid: "948dc57da5194edab5a24f9948b31fd7" instance_seqno: 1786475084698627 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 19:04:44.990276 5860 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 19:04:44.990357 6108 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5b1cb60521c74879877565e7d56818ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b78258430ff4796945bf0d4661a8e63" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 43309 } } peers { permanent_uuid: "948dc57da5194edab5a24f9948b31fd7" member_type: VOTER last_known_addr { host: "127.13.109.61" port: 37729 } } peers { permanent_uuid: "5b1cb60521c74879877565e7d56818ed" member_type: VOTER last_known_addr { host: "127.13.109.60" port: 44399 } }
I20260811 19:04:44.990633 6108 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5b1cb60521c74879877565e7d56818ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b1cb60521c74879877565e7d56818ed, State: Initialized, Role: FOLLOWER
I20260811 19:04:44.990833 5910 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5b78258430ff4796945bf0d4661a8e63 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b78258430ff4796945bf0d4661a8e63" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 43309 } } peers { permanent_uuid: "948dc57da5194edab5a24f9948b31fd7" member_type: VOTER last_known_addr { host: "127.13.109.61" port: 37729 } } peers { permanent_uuid: "5b1cb60521c74879877565e7d56818ed" member_type: VOTER last_known_addr { host: "127.13.109.60" port: 44399 } }
I20260811 19:04:44.991005 6108 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5b1cb60521c74879877565e7d56818ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b78258430ff4796945bf0d4661a8e63" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 43309 } } peers { permanent_uuid: "948dc57da5194edab5a24f9948b31fd7" member_type: VOTER last_known_addr { host: "127.13.109.61" port: 37729 } } peers { permanent_uuid: "5b1cb60521c74879877565e7d56818ed" member_type: VOTER last_known_addr { host: "127.13.109.60" port: 44399 } }
I20260811 19:04:44.991545 6131 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b1cb60521c74879877565e7d56818ed [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b78258430ff4796945bf0d4661a8e63" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 43309 } } peers { permanent_uuid: "948dc57da5194edab5a24f9948b31fd7" member_type: VOTER last_known_addr { host: "127.13.109.61" port: 37729 } } peers { permanent_uuid: "5b1cb60521c74879877565e7d56818ed" member_type: VOTER last_known_addr { host: "127.13.109.60" port: 44399 } } }
I20260811 19:04:44.991768 6131 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b1cb60521c74879877565e7d56818ed [sys.catalog]: This master's current role is: FOLLOWER
I20260811 19:04:44.992136 5910 leader_election.cc:288] T 00000000000000000000000000000000 P 5b78258430ff4796945bf0d4661a8e63 [CANDIDATE]: Term 1 election: Requested vote from peers 948dc57da5194edab5a24f9948b31fd7 (127.13.109.61:37729), 5b1cb60521c74879877565e7d56818ed (127.13.109.60:44399)
I20260811 19:04:44.992514 6108 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5b1cb60521c74879877565e7d56818ed [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:04:44.992719 6079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5b78258430ff4796945bf0d4661a8e63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "5b1cb60521c74879877565e7d56818ed"
I20260811 19:04:44.992929 6079 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5b1cb60521c74879877565e7d56818ed [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:04:44.993497 6020 raft_consensus.cc:348] T 00000000000000000000000000000000 P 948dc57da5194edab5a24f9948b31fd7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b78258430ff4796945bf0d4661a8e63" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 43309 } } peers { permanent_uuid: "948dc57da5194edab5a24f9948b31fd7" member_type: VOTER last_known_addr { host: "127.13.109.61" port: 37729 } } peers { permanent_uuid: "5b1cb60521c74879877565e7d56818ed" member_type: VOTER last_known_addr { host: "127.13.109.60" port: 44399 } }
I20260811 19:04:44.993714 6020 raft_consensus.cc:729] T 00000000000000000000000000000000 P 948dc57da5194edab5a24f9948b31fd7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 948dc57da5194edab5a24f9948b31fd7, State: Initialized, Role: FOLLOWER
I20260811 19:04:44.994086 6020 consensus_queue.cc:260] T 00000000000000000000000000000000 P 948dc57da5194edab5a24f9948b31fd7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b78258430ff4796945bf0d4661a8e63" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 43309 } } peers { permanent_uuid: "948dc57da5194edab5a24f9948b31fd7" member_type: VOTER last_known_addr { host: "127.13.109.61" port: 37729 } } peers { permanent_uuid: "5b1cb60521c74879877565e7d56818ed" member_type: VOTER last_known_addr { host: "127.13.109.60" port: 44399 } }
I20260811 19:04:44.994442 6079 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5b1cb60521c74879877565e7d56818ed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b78258430ff4796945bf0d4661a8e63 in term 1.
I20260811 19:04:44.994931 5885 leader_election.cc:302] T 00000000000000000000000000000000 P 5b78258430ff4796945bf0d4661a8e63 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b1cb60521c74879877565e7d56818ed, 5b78258430ff4796945bf0d4661a8e63; no voters:
I20260811 19:04:44.995296 6117 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b78258430ff4796945bf0d4661a8e63 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b78258430ff4796945bf0d4661a8e63" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 43309 } } peers { permanent_uuid: "948dc57da5194edab5a24f9948b31fd7" member_type: VOTER last_known_addr { host: "127.13.109.61" port: 37729 } } peers { permanent_uuid: "5b1cb60521c74879877565e7d56818ed" member_type: VOTER last_known_addr { host: "127.13.109.60" port: 44399 } } }
I20260811 19:04:44.995440 6117 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b78258430ff4796945bf0d4661a8e63 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 19:04:44.995537 6117 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5b78258430ff4796945bf0d4661a8e63 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:04:44.995620 6117 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5b78258430ff4796945bf0d4661a8e63 [term 1 LEADER]: Becoming Leader. State: Replica: 5b78258430ff4796945bf0d4661a8e63, State: Running, Role: LEADER
I20260811 19:04:44.995944 6117 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5b78258430ff4796945bf0d4661a8e63 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b78258430ff4796945bf0d4661a8e63" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 43309 } } peers { permanent_uuid: "948dc57da5194edab5a24f9948b31fd7" member_type: VOTER last_known_addr { host: "127.13.109.61" port: 37729 } } peers { permanent_uuid: "5b1cb60521c74879877565e7d56818ed" member_type: VOTER last_known_addr { host: "127.13.109.60" port: 44399 } }
I20260811 19:04:44.996467 6020 sys_catalog.cc:564] T 00000000000000000000000000000000 P 948dc57da5194edab5a24f9948b31fd7 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:04:44.996783 5991 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5b78258430ff4796945bf0d4661a8e63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "948dc57da5194edab5a24f9948b31fd7"
I20260811 19:04:44.997018 5991 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 948dc57da5194edab5a24f9948b31fd7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:04:44.997980 6117 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b78258430ff4796945bf0d4661a8e63 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5b78258430ff4796945bf0d4661a8e63. Latest consensus state: current_term: 1 leader_uuid: "5b78258430ff4796945bf0d4661a8e63" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b78258430ff4796945bf0d4661a8e63" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 43309 } } peers { permanent_uuid: "948dc57da5194edab5a24f9948b31fd7" member_type: VOTER last_known_addr { host: "127.13.109.61" port: 37729 } } peers { permanent_uuid: "5b1cb60521c74879877565e7d56818ed" member_type: VOTER last_known_addr { host: "127.13.109.60" port: 44399 } } }
I20260811 19:04:44.998082 6117 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b78258430ff4796945bf0d4661a8e63 [sys.catalog]: This master's current role is: LEADER
I20260811 19:04:44.998545 5991 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 948dc57da5194edab5a24f9948b31fd7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b78258430ff4796945bf0d4661a8e63 in term 1.
I20260811 19:04:45.003842 6141 sys_catalog.cc:455] T 00000000000000000000000000000000 P 948dc57da5194edab5a24f9948b31fd7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b78258430ff4796945bf0d4661a8e63" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 43309 } } peers { permanent_uuid: "948dc57da5194edab5a24f9948b31fd7" member_type: VOTER last_known_addr { host: "127.13.109.61" port: 37729 } } peers { permanent_uuid: "5b1cb60521c74879877565e7d56818ed" member_type: VOTER last_known_addr { host: "127.13.109.60" port: 44399 } } }
I20260811 19:04:45.003981 6141 sys_catalog.cc:458] T 00000000000000000000000000000000 P 948dc57da5194edab5a24f9948b31fd7 [sys.catalog]: This master's current role is: FOLLOWER
W20260811 19:04:45.007819 6170 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 948dc57da5194edab5a24f9948b31fd7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 19:04:45.007920 6170 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260811 19:04:45.011200 6149 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5b1cb60521c74879877565e7d56818ed: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 19:04:45.011307 6149 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 19:04:45.014460 6114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:44.994756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57354 (local address 127.13.109.60:44399)
0811 19:04:44.994868 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:44.994873 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:44.997317 (+ 2444us) server_negotiation.cc:407] Connection header received
0811 19:04:44.997576 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:44.997582 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:44.997642 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:44.997700 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:44.998844 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:44.999598 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:45.000770 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:45.001099 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:45.001178 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:45.010408 (+ 9230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:04:45.010412 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:04:45.010415 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:04:45.010446 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:04:45.012292 (+ 1846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:45.012858 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:45.012862 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:45.012864 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:45.012972 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:45.013323 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:45.013327 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:45.013329 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:45.013570 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:04:45.013715 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:45.014127 (+ 412us) server_negotiation.cc:299] Negotiation successful
0811 19:04:45.014284 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 19:04:45.015918 6162 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:04:45.017122 6162 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:04:45.019443 6028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:44.996697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44176 (local address 127.13.109.61:37729)
0811 19:04:44.996815 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:44.996820 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:45.004325 (+ 7505us) server_negotiation.cc:407] Connection header received
0811 19:04:45.004546 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:45.004550 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:45.004596 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:45.004646 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:45.005998 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:45.006715 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:45.007783 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:45.008494 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:45.008593 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:45.015116 (+ 6523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:04:45.015120 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:04:45.015123 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:04:45.015155 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:04:45.017410 (+ 2255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:45.017980 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:45.017983 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:45.017985 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:45.018079 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:45.018388 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:45.018392 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:45.018394 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:45.018626 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:04:45.018758 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:45.019129 (+ 371us) server_negotiation.cc:299] Negotiation successful
0811 19:04:45.019270 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 19:04:45.020066 5941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:44.996032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43282 (local address 127.13.109.62:43309)
0811 19:04:44.996135 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:44.996140 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:45.002217 (+ 6077us) server_negotiation.cc:407] Connection header received
0811 19:04:45.002405 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:45.002409 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:45.002457 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:45.002509 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:45.003731 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:45.004498 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:45.006974 (+ 2476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:45.007343 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:45.007424 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:45.016326 (+ 8902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:04:45.016333 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:04:45.016335 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:04:45.016365 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:04:45.018168 (+ 1803us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:45.018671 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:45.018674 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:45.018676 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:45.018764 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:45.019076 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:45.019079 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:45.019081 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:45.019287 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:04:45.019392 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:45.019803 (+ 411us) server_negotiation.cc:299] Negotiation successful
0811 19:04:45.019933 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260811 19:04:45.024708 6079 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5b1cb60521c74879877565e7d56818ed [term 1 FOLLOWER]: Refusing update from remote peer 5b78258430ff4796945bf0d4661a8e63: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:04:45.025496 6150 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5b78258430ff4796945bf0d4661a8e63 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b1cb60521c74879877565e7d56818ed" member_type: VOTER last_known_addr { host: "127.13.109.60" port: 44399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:04:45.026028 5991 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 948dc57da5194edab5a24f9948b31fd7 [term 1 FOLLOWER]: Refusing update from remote peer 5b78258430ff4796945bf0d4661a8e63: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:04:45.030404 6131 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b1cb60521c74879877565e7d56818ed [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5b78258430ff4796945bf0d4661a8e63. Latest consensus state: current_term: 1 leader_uuid: "5b78258430ff4796945bf0d4661a8e63" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b78258430ff4796945bf0d4661a8e63" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 43309 } } peers { permanent_uuid: "948dc57da5194edab5a24f9948b31fd7" member_type: VOTER last_known_addr { host: "127.13.109.61" port: 37729 } } peers { permanent_uuid: "5b1cb60521c74879877565e7d56818ed" member_type: VOTER last_known_addr { host: "127.13.109.60" port: 44399 } } }
I20260811 19:04:45.030573 6131 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b1cb60521c74879877565e7d56818ed [sys.catalog]: This master's current role is: FOLLOWER
I20260811 19:04:45.030795 6150 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5b78258430ff4796945bf0d4661a8e63 [LEADER]: Connected to new peer: Peer: permanent_uuid: "948dc57da5194edab5a24f9948b31fd7" member_type: VOTER last_known_addr { host: "127.13.109.61" port: 37729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:04:45.034250 6150 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b78258430ff4796945bf0d4661a8e63 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5b78258430ff4796945bf0d4661a8e63" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b78258430ff4796945bf0d4661a8e63" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 43309 } } peers { permanent_uuid: "948dc57da5194edab5a24f9948b31fd7" member_type: VOTER last_known_addr { host: "127.13.109.61" port: 37729 } } peers { permanent_uuid: "5b1cb60521c74879877565e7d56818ed" member_type: VOTER last_known_addr { host: "127.13.109.60" port: 44399 } } }
I20260811 19:04:45.034425 6150 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b78258430ff4796945bf0d4661a8e63 [sys.catalog]: This master's current role is: LEADER
I20260811 19:04:45.034583 6131 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b1cb60521c74879877565e7d56818ed [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5b78258430ff4796945bf0d4661a8e63" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b78258430ff4796945bf0d4661a8e63" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 43309 } } peers { permanent_uuid: "948dc57da5194edab5a24f9948b31fd7" member_type: VOTER last_known_addr { host: "127.13.109.61" port: 37729 } } peers { permanent_uuid: "5b1cb60521c74879877565e7d56818ed" member_type: VOTER last_known_addr { host: "127.13.109.60" port: 44399 } } }
I20260811 19:04:45.034693 6131 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b1cb60521c74879877565e7d56818ed [sys.catalog]: This master's current role is: FOLLOWER
I20260811 19:04:45.034776 6175 mvcc.cc:205] Tried to move back new op lower bound from 7317401948253728768 to 7317401948147331072. Current Snapshot: MvccSnapshot[applied={T|T < 7317401948253728768}]
I20260811 19:04:45.035094 6174 mvcc.cc:205] Tried to move back new op lower bound from 7317401948253728768 to 7317401948147331072. Current Snapshot: MvccSnapshot[applied={T|T < 7317401948253728768}]
I20260811 19:04:45.035624 6141 sys_catalog.cc:455] T 00000000000000000000000000000000 P 948dc57da5194edab5a24f9948b31fd7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5b78258430ff4796945bf0d4661a8e63. Latest consensus state: current_term: 1 leader_uuid: "5b78258430ff4796945bf0d4661a8e63" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b78258430ff4796945bf0d4661a8e63" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 43309 } } peers { permanent_uuid: "948dc57da5194edab5a24f9948b31fd7" member_type: VOTER last_known_addr { host: "127.13.109.61" port: 37729 } } peers { permanent_uuid: "5b1cb60521c74879877565e7d56818ed" member_type: VOTER last_known_addr { host: "127.13.109.60" port: 44399 } } }
I20260811 19:04:45.035795 6141 sys_catalog.cc:458] T 00000000000000000000000000000000 P 948dc57da5194edab5a24f9948b31fd7 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 19:04:45.036976 6117 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b78258430ff4796945bf0d4661a8e63 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5b78258430ff4796945bf0d4661a8e63" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b78258430ff4796945bf0d4661a8e63" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 43309 } } peers { permanent_uuid: "948dc57da5194edab5a24f9948b31fd7" member_type: VOTER last_known_addr { host: "127.13.109.61" port: 37729 } } peers { permanent_uuid: "5b1cb60521c74879877565e7d56818ed" member_type: VOTER last_known_addr { host: "127.13.109.60" port: 44399 } } }
I20260811 19:04:45.037115 6117 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b78258430ff4796945bf0d4661a8e63 [sys.catalog]: This master's current role is: LEADER
I20260811 19:04:45.038182 6141 sys_catalog.cc:455] T 00000000000000000000000000000000 P 948dc57da5194edab5a24f9948b31fd7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5b78258430ff4796945bf0d4661a8e63" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b78258430ff4796945bf0d4661a8e63" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 43309 } } peers { permanent_uuid: "948dc57da5194edab5a24f9948b31fd7" member_type: VOTER last_known_addr { host: "127.13.109.61" port: 37729 } } peers { permanent_uuid: "5b1cb60521c74879877565e7d56818ed" member_type: VOTER last_known_addr { host: "127.13.109.60" port: 44399 } } }
I20260811 19:04:45.038318 6141 sys_catalog.cc:458] T 00000000000000000000000000000000 P 948dc57da5194edab5a24f9948b31fd7 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 19:04:45.038889 6162 catalog_manager.cc:1347] Generated new cluster ID: 1a4aee8606d74526a50a730d7e9e4020
I20260811 19:04:45.038959 6162 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:04:45.040479 6177 mvcc.cc:205] Tried to move back new op lower bound from 7317401948253728768 to 7317401948147331072. Current Snapshot: MvccSnapshot[applied={T|T < 7317401948253728768}]
I20260811 19:04:45.053699 6162 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:04:45.055135 6162 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:04:45.071125 6162 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5b78258430ff4796945bf0d4661a8e63: Generated new TSK 0
I20260811 19:04:45.072194 6162 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 19:04:45.105119 6114 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:04:45.102405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57370 (local address 127.13.109.60:44399)
0811 19:04:45.102539 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:45.102544 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:45.103775 (+ 1231us) server_negotiation.cc:407] Connection header received
0811 19:04:45.103982 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:45.103986 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:45.104048 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:45.104137 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:45.104946 (+ 809us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57370: BlockingRecv error: recv got EOF from 127.0.0.1:57370 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260811 19:04:45.105341 5941 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:04:45.101744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43286 (local address 127.13.109.62:43309)
0811 19:04:45.101944 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:45.101949 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:45.104173 (+ 2224us) server_negotiation.cc:407] Connection header received
0811 19:04:45.104435 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:45.104438 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:45.104497 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:45.104554 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:45.105193 (+ 639us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43286: BlockingRecv error: recv got EOF from 127.0.0.1:43286 (error 108)
Metrics: {"server-negotiator.queue_time_us":109}
W20260811 19:04:45.106645 6028 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:04:45.103119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44182 (local address 127.13.109.61:37729)
0811 19:04:45.103254 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:45.103259 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:45.105673 (+ 2414us) server_negotiation.cc:407] Connection header received
0811 19:04:45.105908 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:45.105911 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:45.105965 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:45.106020 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:45.106527 (+ 507us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44182: BlockingRecv error: recv got EOF from 127.0.0.1:44182 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260811 19:04:45.111354 6028 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:04:45.109651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44196 (local address 127.13.109.61:37729)
0811 19:04:45.109772 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:45.109776 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:45.110342 (+ 566us) server_negotiation.cc:407] Connection header received
0811 19:04:45.110572 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:45.110577 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:45.110664 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:45.110734 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:45.111248 (+ 514us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44196: BlockingRecv error: recv got EOF from 127.0.0.1:44196 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260811 19:04:45.120680 6028 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:04:45.118746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44202 (local address 127.13.109.61:37729)
0811 19:04:45.118880 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:45.118885 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:45.119563 (+ 678us) server_negotiation.cc:407] Connection header received
0811 19:04:45.119820 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:45.119824 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:45.119885 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:45.119941 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:45.120554 (+ 613us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44202: BlockingRecv error: recv got EOF from 127.0.0.1:44202 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260811 19:04:45.127379 6028 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:04:45.125285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44210 (local address 127.13.109.61:37729)
0811 19:04:45.125408 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:45.125412 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:45.126097 (+ 685us) server_negotiation.cc:407] Connection header received
0811 19:04:45.126314 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:45.126318 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:45.126375 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:45.126431 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:45.127258 (+ 827us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44210: BlockingRecv error: recv got EOF from 127.0.0.1:44210 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260811 19:04:45.135960 6028 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:04:45.133955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44212 (local address 127.13.109.61:37729)
0811 19:04:45.134127 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:45.134132 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:45.134814 (+ 682us) server_negotiation.cc:407] Connection header received
0811 19:04:45.135074 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:45.135080 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:45.135157 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:45.135226 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:45.135835 (+ 609us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44212: BlockingRecv error: recv got EOF from 127.0.0.1:44212 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
W20260811 19:04:45.148792 6028 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:04:45.146953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44220 (local address 127.13.109.61:37729)
0811 19:04:45.147081 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:45.147087 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:45.147728 (+ 641us) server_negotiation.cc:407] Connection header received
0811 19:04:45.147961 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:45.147965 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:45.148025 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:45.148083 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:45.148648 (+ 565us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44220: BlockingRecv error: recv got EOF from 127.0.0.1:44220 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260811 19:04:45.169516 6028 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:04:45.167684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44232 (local address 127.13.109.61:37729)
0811 19:04:45.167835 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:45.167842 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:45.168421 (+ 579us) server_negotiation.cc:407] Connection header received
0811 19:04:45.168650 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:45.168654 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:45.168724 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:45.168781 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:45.169394 (+ 613us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44232: BlockingRecv error: recv got EOF from 127.0.0.1:44232 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
W20260811 19:04:45.206485 6028 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:04:45.204524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44240 (local address 127.13.109.61:37729)
0811 19:04:45.204707 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:45.204713 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:45.205355 (+ 642us) server_negotiation.cc:407] Connection header received
0811 19:04:45.205582 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:45.205586 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:45.205659 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:45.205719 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:45.206352 (+ 633us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44240: BlockingRecv error: recv got EOF from 127.0.0.1:44240 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
W20260811 19:04:45.274564 6028 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:04:45.273290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44244 (local address 127.13.109.61:37729)
0811 19:04:45.273473 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:45.273477 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:45.273597 (+ 120us) server_negotiation.cc:407] Connection header received
0811 19:04:45.273735 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:45.273738 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:45.273844 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:45.273921 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:45.274433 (+ 512us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44244: BlockingRecv error: recv got EOF from 127.0.0.1:44244 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
W20260811 19:04:45.406340 6028 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:04:45.404919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44250 (local address 127.13.109.61:37729)
0811 19:04:45.405077 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:45.405083 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:45.405298 (+ 215us) server_negotiation.cc:407] Connection header received
0811 19:04:45.405426 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:45.405429 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:45.405478 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:45.405552 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:45.406213 (+ 661us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44250: BlockingRecv error: recv got EOF from 127.0.0.1:44250 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260811 19:04:45.666172 6028 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:04:45.664828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44256 (local address 127.13.109.61:37729)
0811 19:04:45.664997 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:45.665001 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:45.665033 (+ 32us) server_negotiation.cc:407] Connection header received
0811 19:04:45.665261 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:45.665264 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:45.665320 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:45.665407 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:45.666040 (+ 633us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44256: BlockingRecv error: recv got EOF from 127.0.0.1:44256 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 19:04:46.009371 6170 catalog_manager.cc:1259] Loaded cluster ID: 1a4aee8606d74526a50a730d7e9e4020
I20260811 19:04:46.009480 6170 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 948dc57da5194edab5a24f9948b31fd7: loading cluster ID for follower catalog manager: success
I20260811 19:04:46.011286 6170 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 948dc57da5194edab5a24f9948b31fd7: acquiring CA information for follower catalog manager: success
I20260811 19:04:46.012614 6149 catalog_manager.cc:1259] Loaded cluster ID: 1a4aee8606d74526a50a730d7e9e4020
I20260811 19:04:46.012708 6149 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5b1cb60521c74879877565e7d56818ed: loading cluster ID for follower catalog manager: success
I20260811 19:04:46.013068 6170 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 948dc57da5194edab5a24f9948b31fd7: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 19:04:46.014621 6149 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5b1cb60521c74879877565e7d56818ed: acquiring CA information for follower catalog manager: success
I20260811 19:04:46.015847 6149 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 5b1cb60521c74879877565e7d56818ed: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 19:04:46.185241 6253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:46.180572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44260 (local address 127.13.109.61:37729)
0811 19:04:46.180855 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:46.180860 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:46.180937 (+ 77us) server_negotiation.cc:407] Connection header received
0811 19:04:46.181103 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:46.181105 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:46.181267 (+ 162us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:46.181426 (+ 159us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:04:46.181835 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:46.182450 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:46.183695 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:46.183936 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:46.184020 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:46.184320 (+ 300us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:04:46.184788 (+ 468us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:04:46.184882 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:46.184949 (+ 67us) server_negotiation.cc:299] Negotiation successful
0811 19:04:46.185002 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":77,"threads_started":1}
I20260811 19:04:46.192677 6259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:46.188656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57374 (local address 127.13.109.60:44399)
0811 19:04:46.188933 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:46.188938 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:46.188954 (+ 16us) server_negotiation.cc:407] Connection header received
0811 19:04:46.189153 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:46.189155 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:46.189283 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:46.189417 (+ 134us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:04:46.189665 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:46.190214 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:46.191085 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:46.191316 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:46.191428 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:46.191756 (+ 328us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:04:46.192221 (+ 465us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:04:46.192388 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:46.192422 (+ 34us) server_negotiation.cc:299] Negotiation successful
0811 19:04:46.192479 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":95,"threads_started":1}
I20260811 19:04:46.200382 6259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:46.196306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57378 (local address 127.13.109.60:44399)
0811 19:04:46.196443 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:46.196446 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:46.196458 (+ 12us) server_negotiation.cc:407] Connection header received
0811 19:04:46.196630 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:46.196633 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:46.196704 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:46.196775 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:04:46.197182 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:46.198014 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:46.199267 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:46.199510 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:46.199628 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:46.199906 (+ 278us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:04:46.199988 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:04:46.200076 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:46.200166 (+ 90us) server_negotiation.cc:299] Negotiation successful
0811 19:04:46.200220 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 19:04:46.201967 6265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:46.195963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43294 (local address 127.13.109.62:43309)
0811 19:04:46.196370 (+ 407us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:46.196376 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:46.197112 (+ 736us) server_negotiation.cc:407] Connection header received
0811 19:04:46.197251 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:46.197254 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:46.197477 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:46.197656 (+ 179us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:04:46.197936 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:46.198771 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:46.200310 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:46.200680 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:46.200785 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:46.201072 (+ 287us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:04:46.201481 (+ 409us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:04:46.201593 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:46.201668 (+ 75us) server_negotiation.cc:299] Negotiation successful
0811 19:04:46.201745 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":286,"thread_start_us":125,"threads_started":1}
I20260811 19:04:46.202101 6253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:46.196903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44262 (local address 127.13.109.61:37729)
0811 19:04:46.197049 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:46.197055 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:46.197242 (+ 187us) server_negotiation.cc:407] Connection header received
0811 19:04:46.197456 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:46.197460 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:46.197525 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:46.197581 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:04:46.198057 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:46.198604 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:46.200310 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:46.200675 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:46.200769 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:46.201500 (+ 731us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:04:46.201596 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:04:46.201651 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:46.201879 (+ 228us) server_negotiation.cc:299] Negotiation successful
0811 19:04:46.201944 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 19:04:46.205641 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 5860
I20260811 19:04:46.214946 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 5946
I20260811 19:04:46.224159 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 6029
2026-08-11T19:04:46Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3222 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474792496492-13748-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:04:46 dist-test-slave-0kls krb5kdc[6270](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:04:46 dist-test-slave-0kls krb5kdc[6270](info): set up 2 sockets
Aug 11 19:04:46 dist-test-slave-0kls krb5kdc[6270](info): commencing operation
krb5kdc: starting...
W20260811 19:04:47.371994 13748 mini_kdc.cc:121] Time spent starting KDC: real 1.123s user 0.005s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474792496492-13748-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474792496492-13748-0/krb5kdc/test-user.keytab.
Aug 11 19:04:47 dist-test-slave-0kls krb5kdc[6270](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475087, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:04:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:04:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.109.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.109.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.109.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab.
Entry for principal kudu/127.13.109.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab.
I20260811 19:04:47.513055 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474792496492-13748-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474792496492-13748-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474792496492-13748-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.62:42667
--webserver_interface=127.13.109.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab
--principal=kudu/127.13.109.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:46353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.109.62:42667,127.13.109.61:36705,127.13.109.60:39453
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:04:47.631037 6284 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 19:04:47.631379 6284 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:04:47.631407 6284 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:04:47.631522 6284 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:04:47.635751 6284 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:04:47.635842 6284 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:04:47.635862 6284 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:04:47.635879 6284 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:04:47.635895 6284 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:04:47.641760 6284 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:46353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474792496492-13748-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474792496492-13748-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.109.62:42667,127.13.109.61:36705,127.13.109.60:39453
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.62.keytab
--principal=kudu/127.13.109.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.62:42667
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.109.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6284
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474792496492-13748-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:04:47.643150 6284 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:04:47.644294 6284 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:04:47.654207 6292 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:04:47.654209 6289 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:04:47.654217 6290 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:04:47.654855 6284 server_base.cc:1034] running on GCE node
I20260811 19:04:47.655674 6284 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:04:47.656996 6284 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:04:47.658185 6284 hybrid_clock.cc:648] HybridClock initialized: now 1786475087658170 us; error 31 us; skew 500 ppm
Aug 11 19:04:47 dist-test-slave-0kls krb5kdc[6270](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475087, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:04:47.661343 6284 init.cc:377] Logged in from keytab as kudu/127.13.109.62@KRBTEST.COM (short username kudu)
I20260811 19:04:47.662544 6284 webserver.cc:460] Webserver started at http://127.13.109.62:39253/ using document root <none> and password file <none>
I20260811 19:04:47.663174 6284 fs_manager.cc:359] Metadata directory not provided
I20260811 19:04:47.663245 6284 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:04:47.663444 6284 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:04:47.666234 6284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474792496492-13748-0/minicluster-data/master-0/data/instance:
uuid: "3cfcc2b5054946a3b031616c64ee1801"
format_stamp: "Formatted at 2026-08-11 19:04:47 on dist-test-slave-0kls"
server_key: "5cf794bf0506c60c99e30d89704cd4d1"
server_key_iv: "c698b6be9c5fcf84f8493d87aeda8581"
server_key_version: "encryptionkey@0"
I20260811 19:04:47.666865 6284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474792496492-13748-0/minicluster-data/master-0/wal/instance:
uuid: "3cfcc2b5054946a3b031616c64ee1801"
format_stamp: "Formatted at 2026-08-11 19:04:47 on dist-test-slave-0kls"
server_key: "5cf794bf0506c60c99e30d89704cd4d1"
server_key_iv: "c698b6be9c5fcf84f8493d87aeda8581"
server_key_version: "encryptionkey@0"
I20260811 19:04:47.671171 6284 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260811 19:04:47.673985 6300 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:04:47.675350 6284 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 19:04:47.675516 6284 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474792496492-13748-0/minicluster-data/master-0/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474792496492-13748-0/minicluster-data/master-0/wal
uuid: "3cfcc2b5054946a3b031616c64ee1801"
format_stamp: "Formatted at 2026-08-11 19:04:47 on dist-test-slave-0kls"
server_key: "5cf794bf0506c60c99e30d89704cd4d1"
server_key_iv: "c698b6be9c5fcf84f8493d87aeda8581"
server_key_version: "encryptionkey@0"
I20260811 19:04:47.675650 6284 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474792496492-13748-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474792496492-13748-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474792496492-13748-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:04:47.698021 6284 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:04:47.701433 6284 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:04:47.701686 6284 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:04:47.709424 6284 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.62:42667
I20260811 19:04:47.709758 6356 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.62:42667 every 8 connection(s)
I20260811 19:04:47.710656 6284 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474792496492-13748-0/minicluster-data/master-0/data/info.pb
I20260811 19:04:47.711020 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 6284
I20260811 19:04:47.711118 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474792496492-13748-0/minicluster-data/master-0/wal/instance
I20260811 19:04:47.711517 13748 external_mini_cluster.cc:1428] Setting key 76ddbe952f2cec26b3c927a35a66fefb
I20260811 19:04:47.713847 6357 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:04:47.719877 6357 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.109.62" port: 42667 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.13.109.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.109.61@KRBTEST.COM" created.
Aug 11 19:04:47 dist-test-slave-0kls krb5kdc[6270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475087, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.62@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
I20260811 19:04:47.739866 6360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:47.723057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:42667 (local address 127.0.0.1:46654)
0811 19:04:47.723663 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:47.723713 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 19:04:47.724587 (+ 874us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:04:47.725505 (+ 918us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:04:47.725510 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:04:47.726097 (+ 587us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:04:47.726412 (+ 315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:47.726426 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:47.728281 (+ 1855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:47.728288 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:47.729767 (+ 1479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:47.729813 (+ 46us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:47.730572 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:47.730581 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:47.730755 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:47.731497 (+ 742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:04:47.731533 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:04:47.733592 (+ 2059us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:04:47.736485 (+ 2893us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:04:47.736494 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:04:47.736503 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:04:47.736752 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:04:47.737231 (+ 479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:04:47.737234 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:04:47.737235 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:04:47.737356 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:04:47.738271 (+ 915us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:04:47.738277 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:04:47.738418 (+ 141us) client_negotiation.cc:769] Sending connection context
0811 19:04:47.738626 (+ 208us) client_negotiation.cc:240] Negotiation successful
0811 19:04:47.738923 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":138,"threads_started":1}
I20260811 19:04:47.739871 6361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:47.723977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46654 (local address 127.13.109.62:42667)
0811 19:04:47.724293 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:47.724302 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:47.724342 (+ 40us) server_negotiation.cc:407] Connection header received
0811 19:04:47.724997 (+ 655us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:47.725003 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:47.725220 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:47.725387 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:47.726600 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:47.728076 (+ 1476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:47.729950 (+ 1874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:47.730362 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:47.730506 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:47.733818 (+ 3312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:04:47.733833 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:04:47.733856 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:04:47.733899 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:04:47.736333 (+ 2434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:47.736899 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:47.736910 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:47.736915 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:47.737109 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:47.737471 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:47.737478 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:47.737482 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:47.738141 (+ 659us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:04:47.738282 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:47.738748 (+ 466us) server_negotiation.cc:299] Negotiation successful
0811 19:04:47.739020 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":79,"server-negotiator.queue_time_us":196,"thread_start_us":106,"threads_started":1}
I20260811 19:04:47.742511 6357 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.109.61" port: 36705 } has no permanent_uuid. Determining permanent_uuid...
I20260811 19:04:47.743294 6360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:47.743000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.61:36705 (local address 127.0.0.1:32828)
0811 19:04:47.743155 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:47.743232 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.61:36705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260811 19:04:47.743588 6302 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.109.61:36705: connect: Connection refused (error 111)
I20260811 19:04:47.744539 6360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:47.744339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.61:36705 (local address 127.0.0.1:32836)
0811 19:04:47.744438 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:47.744484 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.61:36705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260811 19:04:47.744738 6357 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.109.61:36705: Network error: Client connection negotiation failed: client connection to 127.13.109.61:36705: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.109.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.61.keytab.
Entry for principal kudu/127.13.109.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.61.keytab.
I20260811 19:04:47.746780 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474792496492-13748-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474792496492-13748-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474792496492-13748-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474792496492-13748-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.61:36705
--webserver_interface=127.13.109.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.61.keytab
--principal=kudu/127.13.109.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:46353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.109.62:42667,127.13.109.61:36705,127.13.109.60:39453
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 19:04:47.794025 6357 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.109.61" port: 36705 } attempt: 1
I20260811 19:04:47.795046 6360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:47.794696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.61:36705 (local address 127.0.0.1:32850)
0811 19:04:47.794891 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:47.794959 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.61:36705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260811 19:04:47.795867 6360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:47.795633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.61:36705 (local address 127.0.0.1:32854)
0811 19:04:47.795748 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:47.795805 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.61:36705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260811 19:04:47.796005 6357 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.109.61:36705: Network error: Client connection negotiation failed: client connection to 127.13.109.61:36705: connect: Connection refused (error 111)
I20260811 19:04:47.864208 6357 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.109.61" port: 36705 } attempt: 2
I20260811 19:04:47.865257 6360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:47.864892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.61:36705 (local address 127.0.0.1:32856)
0811 19:04:47.865091 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:47.865172 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.61:36705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260811 19:04:47.866154 6360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:47.865878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.61:36705 (local address 127.0.0.1:32864)
0811 19:04:47.866040 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:47.866090 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.61:36705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
W20260811 19:04:47.866297 6357 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.109.61:36705: Network error: Client connection negotiation failed: client connection to 127.13.109.61:36705: connect: Connection refused (error 111)
W20260811 19:04:47.870859 6365 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 19:04:47.871268 6365 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:04:47.871335 6365 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:04:47.871438 6365 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:04:47.875821 6365 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:04:47.875926 6365 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:04:47.875967 6365 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:04:47.876019 6365 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:04:47.876063 6365 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:04:47.882313 6365 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:46353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474792496492-13748-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474792496492-13748-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.13.109.62:42667,127.13.109.61:36705,127.13.109.60:39453
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.61.keytab
--principal=kudu/127.13.109.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.61:36705
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474792496492-13748-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.13.109.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6365
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474792496492-13748-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:04:47.883654 6365 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:04:47.884788 6365 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 19:04:47.894060 6365 server_base.cc:1034] running on GCE node
W20260811 19:04:47.894048 6370 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:04:47.894025 6373 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:04:47.894017 6371 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:04:47.894944 6365 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:04:47.896241 6365 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:04:47.897411 6365 hybrid_clock.cc:648] HybridClock initialized: now 1786475087897394 us; error 31 us; skew 500 ppm
Aug 11 19:04:47 dist-test-slave-0kls krb5kdc[6270](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475087, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:04:47.900633 6365 init.cc:377] Logged in from keytab as kudu/127.13.109.61@KRBTEST.COM (short username kudu)
I20260811 19:04:47.901707 6365 webserver.cc:460] Webserver started at http://127.13.109.61:38127/ using document root <none> and password file <none>
I20260811 19:04:47.902341 6365 fs_manager.cc:359] Metadata directory not provided
I20260811 19:04:47.902441 6365 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:04:47.902701 6365 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:04:47.904955 6365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474792496492-13748-0/minicluster-data/master-1/data/instance:
uuid: "8113a711c6ff4a2897e089ca7dba484a"
format_stamp: "Formatted at 2026-08-11 19:04:47 on dist-test-slave-0kls"
server_key: "a30e6f708d1c8fd679b33cad9c844f00"
server_key_iv: "a67bd1ca2c322c9f3bad4d4004762846"
server_key_version: "encryptionkey@0"
I20260811 19:04:47.905556 6365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474792496492-13748-0/minicluster-data/master-1/wal/instance:
uuid: "8113a711c6ff4a2897e089ca7dba484a"
format_stamp: "Formatted at 2026-08-11 19:04:47 on dist-test-slave-0kls"
server_key: "a30e6f708d1c8fd679b33cad9c844f00"
server_key_iv: "a67bd1ca2c322c9f3bad4d4004762846"
server_key_version: "encryptionkey@0"
I20260811 19:04:47.909574 6365 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260811 19:04:47.912341 6380 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:04:47.913727 6365 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 19:04:47.913937 6365 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474792496492-13748-0/minicluster-data/master-1/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474792496492-13748-0/minicluster-data/master-1/wal
uuid: "8113a711c6ff4a2897e089ca7dba484a"
format_stamp: "Formatted at 2026-08-11 19:04:47 on dist-test-slave-0kls"
server_key: "a30e6f708d1c8fd679b33cad9c844f00"
server_key_iv: "a67bd1ca2c322c9f3bad4d4004762846"
server_key_version: "encryptionkey@0"
I20260811 19:04:47.914079 6365 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474792496492-13748-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474792496492-13748-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474792496492-13748-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:04:47.949326 6365 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:04:47.952731 6365 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:04:47.952986 6365 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:04:47.957479 6357 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.109.61" port: 36705 } attempt: 3
I20260811 19:04:47.958514 6360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:47.958174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.61:36705 (local address 127.0.0.1:32866)
0811 19:04:47.958312 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:47.958391 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.61:36705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260811 19:04:47.959364 6360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:47.959109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.61:36705 (local address 127.0.0.1:32876)
0811 19:04:47.959236 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:47.959290 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.61:36705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260811 19:04:47.959508 6357 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.109.61:36705: Network error: Client connection negotiation failed: client connection to 127.13.109.61:36705: connect: Connection refused (error 111)
I20260811 19:04:47.961539 6365 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.61:36705
I20260811 19:04:47.961946 6435 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.61:36705 every 8 connection(s)
I20260811 19:04:47.962874 6365 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474792496492-13748-0/minicluster-data/master-1/data/info.pb
I20260811 19:04:47.965570 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 6365
I20260811 19:04:47.965745 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474792496492-13748-0/minicluster-data/master-1/wal/instance
I20260811 19:04:47.966131 13748 external_mini_cluster.cc:1428] Setting key 8924455aa736a5fc53991687b6ae652a
I20260811 19:04:47.966394 6436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:04:47.970894 6436 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.109.62" port: 42667 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.13.109.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.109.60@KRBTEST.COM" created.
Aug 11 19:04:47 dist-test-slave-0kls krb5kdc[6270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475087, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.61@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
I20260811 19:04:47.987474 6361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:47.973239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46660 (local address 127.13.109.62:42667)
0811 19:04:47.973430 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:47.973436 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:47.974509 (+ 1073us) server_negotiation.cc:407] Connection header received
0811 19:04:47.975596 (+ 1087us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:47.975600 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:47.975661 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:47.975749 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:47.977170 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:47.977774 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:47.979671 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:47.979931 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:47.980028 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:47.983001 (+ 2973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:04:47.983008 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:04:47.983015 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:04:47.983060 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:04:47.984830 (+ 1770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:47.985504 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:47.985508 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:47.985512 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:47.985613 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:47.986126 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:47.986129 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:47.986132 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:47.986362 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:04:47.986484 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:47.987134 (+ 650us) server_negotiation.cc:299] Negotiation successful
0811 19:04:47.987287 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 19:04:47.988375 6439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:47.973673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:42667 (local address 127.0.0.1:46660)
0811 19:04:47.974305 (+ 632us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:47.974365 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 19:04:47.975371 (+ 1006us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:04:47.975925 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:04:47.975932 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:04:47.976464 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:04:47.976992 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:47.977008 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:47.978024 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:47.978027 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:47.979532 (+ 1505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:47.979550 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:47.980131 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:47.980138 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:47.980401 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:47.981167 (+ 766us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:04:47.981186 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:04:47.982772 (+ 1586us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:04:47.985001 (+ 2229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:04:47.985009 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:04:47.985032 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:04:47.985373 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:04:47.985730 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:04:47.985733 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:04:47.985735 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:04:47.986007 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:04:47.986484 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:04:47.986493 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:04:47.986829 (+ 336us) client_negotiation.cc:769] Sending connection context
0811 19:04:47.987043 (+ 214us) client_negotiation.cc:240] Negotiation successful
0811 19:04:47.987438 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":164,"threads_started":1}
I20260811 19:04:47.990044 6436 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.109.61" port: 36705 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 19:04:47 dist-test-slave-0kls krb5kdc[6270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475087, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.61@KRBTEST.COM for kudu/127.13.109.61@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.109.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.60.keytab.
Entry for principal kudu/127.13.109.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.60.keytab.
I20260811 19:04:47.997722 13748 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
/tmp/dist-test-taskrGEMHD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474792496492-13748-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474792496492-13748-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474792496492-13748-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474792496492-13748-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.60:39453
--webserver_interface=127.13.109.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.60.keytab
--principal=kudu/127.13.109.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.109.20:46353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.109.62:42667,127.13.109.61:36705,127.13.109.60:39453
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474792496492-13748-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474792496492-13748-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474792496492-13748-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 19:04:48.000648 6442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:47.990743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32880 (local address 127.13.109.61:36705)
0811 19:04:47.991162 (+ 419us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:47.991172 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:47.991192 (+ 20us) server_negotiation.cc:407] Connection header received
0811 19:04:47.991229 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:47.991234 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:47.991393 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:47.991505 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:47.991890 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:47.992733 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:47.993670 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:47.994182 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:47.994272 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:47.996419 (+ 2147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:04:47.996429 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:04:47.996444 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:04:47.996467 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:04:47.998584 (+ 2117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:47.999029 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:47.999039 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:47.999044 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:47.999151 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:47.999419 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:47.999425 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:47.999429 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:47.999855 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:04:47.999970 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:48.000333 (+ 363us) server_negotiation.cc:299] Negotiation successful
0811 19:04:48.000467 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":177,"threads_started":1}
I20260811 19:04:48.000672 6439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:47.990602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.61:36705 (local address 127.0.0.1:32880)
0811 19:04:47.990794 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:47.990809 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 19:04:47.990903 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:04:47.991519 (+ 616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:04:47.991521 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:04:47.991680 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:04:47.991783 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:47.991789 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:47.992846 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:47.992849 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:47.993541 (+ 692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:47.993552 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:47.994295 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:47.994298 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:47.994376 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:47.994842 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:04:47.994879 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:04:47.996272 (+ 1393us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:04:47.998710 (+ 2438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:04:47.998713 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:04:47.998715 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:04:47.998910 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:04:47.999248 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:04:47.999251 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:04:47.999252 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:04:47.999329 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:04:47.999969 (+ 640us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:04:47.999974 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:04:48.000103 (+ 129us) client_negotiation.cc:769] Sending connection context
0811 19:04:48.000231 (+ 128us) client_negotiation.cc:240] Negotiation successful
0811 19:04:48.000399 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110}
I20260811 19:04:48.005471 6436 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.109.60" port: 39453 } has no permanent_uuid. Determining permanent_uuid...
I20260811 19:04:48.006789 6439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:48.006369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.60:39453 (local address 127.0.0.1:35004)
0811 19:04:48.006579 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:48.006686 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.60:39453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
W20260811 19:04:48.007171 6383 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.109.60:39453: connect: Connection refused (error 111)
I20260811 19:04:48.008497 6439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:48.008198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.60:39453 (local address 127.0.0.1:35012)
0811 19:04:48.008342 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:48.008416 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.60:39453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260811 19:04:48.009013 6436 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.109.60:39453: Network error: Client connection negotiation failed: client connection to 127.13.109.60:39453: connect: Connection refused (error 111)
I20260811 19:04:48.058352 6436 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.109.60" port: 39453 } attempt: 1
I20260811 19:04:48.059372 6439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:48.059027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.60:39453 (local address 127.0.0.1:35022)
0811 19:04:48.059218 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:48.059288 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.60:39453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260811 19:04:48.060182 6439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:48.059945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.60:39453 (local address 127.0.0.1:35026)
0811 19:04:48.060064 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:48.060122 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.60:39453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 19:04:48.060320 6436 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.109.60:39453: Network error: Client connection negotiation failed: client connection to 127.13.109.60:39453: connect: Connection refused (error 111)
I20260811 19:04:48.102695 6357 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.109.61" port: 36705 } attempt: 4
Aug 11 19:04:48 dist-test-slave-0kls krb5kdc[6270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475087, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.62@KRBTEST.COM for kudu/127.13.109.61@KRBTEST.COM
I20260811 19:04:48.113082 6360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:48.103379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.61:36705 (local address 127.0.0.1:32886)
0811 19:04:48.103618 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:48.103635 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 19:04:48.103819 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:04:48.104113 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:04:48.104116 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:04:48.104328 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:04:48.104438 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:48.104443 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:48.105997 (+ 1554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:48.106002 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:48.106988 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:48.107000 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:48.107548 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:48.107575 (+ 27us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:48.107703 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:48.108258 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:04:48.108271 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:04:48.109577 (+ 1306us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:04:48.111442 (+ 1865us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:04:48.111445 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:04:48.111448 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:04:48.111663 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:04:48.111956 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:04:48.111959 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:04:48.111961 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:04:48.112052 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:04:48.112506 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:04:48.112509 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:04:48.112621 (+ 112us) client_negotiation.cc:769] Sending connection context
0811 19:04:48.112739 (+ 118us) client_negotiation.cc:240] Negotiation successful
0811 19:04:48.112905 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131}
I20260811 19:04:48.113082 6442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:48.103493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32886 (local address 127.13.109.61:36705)
0811 19:04:48.103628 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:48.103634 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:48.103743 (+ 109us) server_negotiation.cc:407] Connection header received
0811 19:04:48.103949 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:48.103951 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:48.104002 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:48.104093 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:48.104577 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:48.105860 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:48.107146 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:48.107405 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:48.107527 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:48.109748 (+ 2221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:04:48.109751 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:04:48.109754 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:04:48.109823 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:04:48.111315 (+ 1492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:48.111771 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:48.111775 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:48.111777 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:48.111860 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:48.112196 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:48.112198 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:48.112199 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:48.112404 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:04:48.112515 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:48.112794 (+ 279us) server_negotiation.cc:299] Negotiation successful
0811 19:04:48.112918 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 19:04:48.114087 6357 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.109.60" port: 39453 } has no permanent_uuid. Determining permanent_uuid...
I20260811 19:04:48.114887 6360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:48.114630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.60:39453 (local address 127.0.0.1:35036)
0811 19:04:48.114763 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:48.114815 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.60:39453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260811 19:04:48.115593 6360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:48.115379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.60:39453 (local address 127.0.0.1:35040)
0811 19:04:48.115487 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:48.115533 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.60:39453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260811 19:04:48.115751 6357 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.109.60:39453: Network error: Client connection negotiation failed: client connection to 127.13.109.60:39453: connect: Connection refused (error 111)
W20260811 19:04:48.126055 6443 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 19:04:48.126418 6443 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:04:48.126480 6443 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:04:48.126559 6443 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260811 19:04:48.128476 6436 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.109.60" port: 39453 } attempt: 2
I20260811 19:04:48.129293 6439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:48.128998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.60:39453 (local address 127.0.0.1:35044)
0811 19:04:48.129147 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:48.129217 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.60:39453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260811 19:04:48.130084 6439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:48.129863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.60:39453 (local address 127.0.0.1:35052)
0811 19:04:48.129976 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:48.130031 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.60:39453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260811 19:04:48.130221 6436 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.109.60:39453: Network error: Client connection negotiation failed: client connection to 127.13.109.60:39453: connect: Connection refused (error 111)
W20260811 19:04:48.131048 6443 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:04:48.131136 6443 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:04:48.131199 6443 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:04:48.131239 6443 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:04:48.131275 6443 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:04:48.136845 6443 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.109.20:46353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474792496492-13748-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474792496492-13748-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.13.109.62:42667,127.13.109.61:36705,127.13.109.60:39453
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474792496492-13748-0/krb5kdc/kudu_127.13.109.60.keytab
--principal=kudu/127.13.109.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.109.60:39453
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474792496492-13748-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.13.109.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6443
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474792496492-13748-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 19:04:48.138165 6443 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:04:48.139266 6443 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:04:48.148262 6449 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:04:48.148262 6450 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:04:48.148363 6452 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:04:48.148334 6443 server_base.cc:1034] running on GCE node
I20260811 19:04:48.149210 6443 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:04:48.150444 6443 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:04:48.151612 6443 hybrid_clock.cc:648] HybridClock initialized: now 1786475088151600 us; error 29 us; skew 500 ppm
Aug 11 19:04:48 dist-test-slave-0kls krb5kdc[6270](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475088, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:04:48.154458 6443 init.cc:377] Logged in from keytab as kudu/127.13.109.60@KRBTEST.COM (short username kudu)
I20260811 19:04:48.155368 6443 webserver.cc:460] Webserver started at http://127.13.109.60:33519/ using document root <none> and password file <none>
I20260811 19:04:48.155957 6443 fs_manager.cc:359] Metadata directory not provided
I20260811 19:04:48.156051 6443 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:04:48.156301 6443 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:04:48.158506 6443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474792496492-13748-0/minicluster-data/master-2/data/instance:
uuid: "87e9f2c4ab994aac9d538c96bb1457d8"
format_stamp: "Formatted at 2026-08-11 19:04:48 on dist-test-slave-0kls"
server_key: "1100324aef8ac9bcd19ae7e1bf3de540"
server_key_iv: "7e0fffe16ea33fe0b732fb19dc83ef29"
server_key_version: "encryptionkey@0"
I20260811 19:04:48.159085 6443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474792496492-13748-0/minicluster-data/master-2/wal/instance:
uuid: "87e9f2c4ab994aac9d538c96bb1457d8"
format_stamp: "Formatted at 2026-08-11 19:04:48 on dist-test-slave-0kls"
server_key: "1100324aef8ac9bcd19ae7e1bf3de540"
server_key_iv: "7e0fffe16ea33fe0b732fb19dc83ef29"
server_key_version: "encryptionkey@0"
I20260811 19:04:48.163084 6443 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260811 19:04:48.165721 6460 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:04:48.167090 6443 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 19:04:48.167249 6443 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474792496492-13748-0/minicluster-data/master-2/data,/tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474792496492-13748-0/minicluster-data/master-2/wal
uuid: "87e9f2c4ab994aac9d538c96bb1457d8"
format_stamp: "Formatted at 2026-08-11 19:04:48 on dist-test-slave-0kls"
server_key: "1100324aef8ac9bcd19ae7e1bf3de540"
server_key_iv: "7e0fffe16ea33fe0b732fb19dc83ef29"
server_key_version: "encryptionkey@0"
I20260811 19:04:48.167379 6443 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474792496492-13748-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474792496492-13748-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474792496492-13748-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:04:48.174893 6357 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.109.60" port: 39453 } attempt: 1
I20260811 19:04:48.175838 6360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:48.175461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.60:39453 (local address 127.0.0.1:35054)
0811 19:04:48.175691 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:48.175757 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.60:39453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135}
I20260811 19:04:48.176585 6360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:48.176360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.60:39453 (local address 127.0.0.1:35062)
0811 19:04:48.176475 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:48.176530 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.109.60:39453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260811 19:04:48.176730 6357 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.109.60:39453: Network error: Client connection negotiation failed: client connection to 127.13.109.60:39453: connect: Connection refused (error 111)
I20260811 19:04:48.181025 6443 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:04:48.184206 6443 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:04:48.184413 6443 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:04:48.191634 6443 rpc_server.cc:307] RPC server started. Bound to: 127.13.109.60:39453
I20260811 19:04:48.191938 6516 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.109.60:39453 every 8 connection(s)
I20260811 19:04:48.192657 6443 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474792496492-13748-0/minicluster-data/master-2/data/info.pb
I20260811 19:04:48.195637 6517 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:04:48.199395 13748 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu as pid 6443
I20260811 19:04:48.199540 13748 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrGEMHD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474792496492-13748-0/minicluster-data/master-2/wal/instance
I20260811 19:04:48.199852 13748 external_mini_cluster.cc:1428] Setting key 3b2a1860c5a0e396fbb0cdcb9517cf6a
I20260811 19:04:48.200517 6517 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.109.62" port: 42667 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 19:04:48 dist-test-slave-0kls krb5kdc[6270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475088, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.60@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
Aug 11 19:04:48 dist-test-slave-0kls krb5kdc[6270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475087, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.109.60@KRBTEST.COM
I20260811 19:04:48.213071 6361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:48.202615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46670 (local address 127.13.109.62:42667)
0811 19:04:48.202747 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:48.202751 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:48.203018 (+ 267us) server_negotiation.cc:407] Connection header received
0811 19:04:48.203290 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:48.203293 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:48.203350 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:48.203450 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:48.204459 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:48.204981 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:48.206132 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:48.206344 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:48.206426 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:48.209201 (+ 2775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:04:48.209204 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:04:48.209206 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:04:48.209232 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:04:48.210932 (+ 1700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:48.211473 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:48.211477 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:48.211478 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:48.211562 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:48.211909 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:48.211914 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:48.211915 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:48.212127 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:04:48.212239 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:48.212770 (+ 531us) server_negotiation.cc:299] Negotiation successful
0811 19:04:48.212896 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 19:04:48.214012 6520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:48.201286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35078 (local address 127.13.109.60:39453)
0811 19:04:48.201725 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:48.201743 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:48.201812 (+ 69us) server_negotiation.cc:407] Connection header received
0811 19:04:48.202641 (+ 829us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:48.202671 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:48.203094 (+ 423us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:48.203468 (+ 374us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:48.204117 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:48.205176 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:48.206088 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:48.206526 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:48.206648 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:48.209596 (+ 2948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:04:48.209602 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:04:48.209614 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:04:48.209635 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:04:48.211248 (+ 1613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:48.211713 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:48.211718 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:48.211720 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:48.211814 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:48.212130 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:48.212137 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:48.212139 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:48.212533 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:04:48.212766 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:48.212959 (+ 193us) server_negotiation.cc:299] Negotiation successful
0811 19:04:48.213119 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":76,"server-negotiator.queue_time_us":285,"spinlock_wait_cycles":4352,"thread_start_us":110,"threads_started":1}
I20260811 19:04:48.214013 6521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:48.202476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.62:42667 (local address 127.0.0.1:46670)
0811 19:04:48.202865 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:48.202916 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 19:04:48.203079 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:04:48.203514 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:04:48.203519 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:04:48.203953 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:04:48.204290 (+ 337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:48.204304 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:48.205096 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:48.205099 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:48.205980 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:48.205991 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:48.206490 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:48.206494 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:48.206648 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:48.207470 (+ 822us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:04:48.207486 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:04:48.209013 (+ 1527us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:04:48.211054 (+ 2041us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:04:48.211066 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:04:48.211077 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:04:48.211363 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:04:48.211657 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:04:48.211662 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:04:48.211666 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:04:48.211809 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:04:48.212240 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:04:48.212248 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:04:48.212546 (+ 298us) client_negotiation.cc:769] Sending connection context
0811 19:04:48.212727 (+ 181us) client_negotiation.cc:240] Negotiation successful
0811 19:04:48.213048 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"mutex_wait_us":72,"spinlock_wait_cycles":2304,"thread_start_us":103,"threads_started":1}
I20260811 19:04:48.215973 6517 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.109.61" port: 36705 } has no permanent_uuid. Determining permanent_uuid...
I20260811 19:04:48.221367 6436 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.109.60" port: 39453 } attempt: 3
Aug 11 19:04:48 dist-test-slave-0kls krb5kdc[6270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475088, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.60@KRBTEST.COM for kudu/127.13.109.61@KRBTEST.COM
Aug 11 19:04:48 dist-test-slave-0kls krb5kdc[6270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475087, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.109.62@KRBTEST.COM
Aug 11 19:04:48 dist-test-slave-0kls krb5kdc[6270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475087, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.61@KRBTEST.COM for kudu/127.13.109.60@KRBTEST.COM
I20260811 19:04:48.227106 6521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:48.216521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.61:36705 (local address 127.0.0.1:32890)
0811 19:04:48.216699 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:48.216716 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 19:04:48.216793 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:04:48.217120 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:04:48.217123 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:04:48.217331 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:04:48.217491 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:48.217501 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:48.218585 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:48.218588 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:48.219465 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:48.219476 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:48.220090 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:48.220094 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:48.220204 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:48.220791 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:04:48.220810 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:04:48.222982 (+ 2172us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:04:48.224828 (+ 1846us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:04:48.224833 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:04:48.224838 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:04:48.225078 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:04:48.225940 (+ 862us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:04:48.225944 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:04:48.225946 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:04:48.226046 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:04:48.226551 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:04:48.226556 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:04:48.226660 (+ 104us) client_negotiation.cc:769] Sending connection context
0811 19:04:48.226770 (+ 110us) client_negotiation.cc:240] Negotiation successful
0811 19:04:48.226910 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109}
I20260811 19:04:48.227144 6442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:48.216623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32890 (local address 127.13.109.61:36705)
0811 19:04:48.216902 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:48.216908 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:48.216922 (+ 14us) server_negotiation.cc:407] Connection header received
0811 19:04:48.216960 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:48.216963 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:48.217010 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:48.217089 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:48.217627 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:48.218482 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:48.219579 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:48.219946 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:48.220063 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:48.223110 (+ 3047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:04:48.223120 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:04:48.223123 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:04:48.223145 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:04:48.224680 (+ 1535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:48.225183 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:48.225188 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:48.225191 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:48.225282 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:48.226164 (+ 882us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:48.226169 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:48.226174 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:48.226397 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:04:48.226470 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:48.226844 (+ 374us) server_negotiation.cc:299] Negotiation successful
0811 19:04:48.226962 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201}
I20260811 19:04:48.228183 6517 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.109.60" port: 39453 } has no permanent_uuid. Determining permanent_uuid...
I20260811 19:04:48.228580 6361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:48.216759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46684 (local address 127.13.109.62:42667)
0811 19:04:48.216874 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:48.216878 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:48.216891 (+ 13us) server_negotiation.cc:407] Connection header received
0811 19:04:48.216930 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:48.216934 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:48.216979 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:48.217065 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:48.217850 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:48.218604 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:48.219655 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:48.219979 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:48.220072 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:48.223283 (+ 3211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:04:48.223288 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:04:48.223290 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:04:48.223318 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:04:48.225866 (+ 2548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:48.226298 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:48.226302 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:48.226303 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:48.226392 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:48.226724 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:48.226729 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:48.226730 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:48.226967 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:04:48.227568 (+ 601us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:48.228276 (+ 708us) server_negotiation.cc:299] Negotiation successful
0811 19:04:48.228439 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 19:04:48.230466 6439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:48.221879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.60:39453 (local address 127.0.0.1:35088)
0811 19:04:48.222030 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:48.222046 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 19:04:48.222135 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:04:48.222409 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:04:48.222413 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:04:48.222595 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:04:48.222723 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:48.222733 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:48.223720 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:48.223725 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:48.224241 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:48.224249 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:48.224902 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:48.224906 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:48.224991 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:48.225431 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:04:48.225443 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:04:48.226798 (+ 1355us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:04:48.228791 (+ 1993us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:04:48.228797 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:04:48.228800 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:04:48.229063 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:04:48.229362 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:04:48.229367 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:04:48.229369 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:04:48.229464 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:04:48.229955 (+ 491us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:04:48.229962 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:04:48.230061 (+ 99us) client_negotiation.cc:769] Sending connection context
0811 19:04:48.230162 (+ 101us) client_negotiation.cc:240] Negotiation successful
0811 19:04:48.230292 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260811 19:04:48.231154 6520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:48.222063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35088 (local address 127.13.109.60:39453)
0811 19:04:48.222201 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:48.222205 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:48.222219 (+ 14us) server_negotiation.cc:407] Connection header received
0811 19:04:48.222259 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:48.222261 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:48.222294 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:48.222358 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:48.222829 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:48.223610 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:48.224374 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:48.224780 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:48.224899 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:48.226930 (+ 2031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:04:48.226933 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:04:48.226936 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:04:48.226957 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:04:48.228691 (+ 1734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:48.229165 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:48.229168 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:48.229170 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:48.229258 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:48.229571 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:48.229577 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:48.229578 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:48.229834 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:04:48.230774 (+ 940us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:48.230870 (+ 96us) server_negotiation.cc:299] Negotiation successful
0811 19:04:48.230996 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 19:04:48.234602 6436 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8113a711c6ff4a2897e089ca7dba484a: Bootstrap starting.
Aug 11 19:04:48 dist-test-slave-0kls krb5kdc[6270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475088, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.60@KRBTEST.COM for kudu/127.13.109.60@KRBTEST.COM
Aug 11 19:04:48 dist-test-slave-0kls krb5kdc[6270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475087, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.109.61@KRBTEST.COM
I20260811 19:04:48.237905 6436 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8113a711c6ff4a2897e089ca7dba484a: Neither blocks nor log segments found. Creating new log.
I20260811 19:04:48.238705 6521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:48.228776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.60:39453 (local address 127.0.0.1:35098)
0811 19:04:48.228876 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:48.228889 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 19:04:48.228955 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:04:48.230084 (+ 1129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:04:48.230087 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:04:48.230257 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:04:48.230345 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:48.230352 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:48.231824 (+ 1472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:48.231827 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:48.232364 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:48.232373 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:48.232854 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:48.232857 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:48.232954 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:48.233437 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:04:48.233453 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:04:48.235037 (+ 1584us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:04:48.236941 (+ 1904us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:04:48.236945 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:04:48.236949 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:04:48.237178 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:04:48.237477 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:04:48.237480 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:04:48.237482 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:04:48.237567 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:04:48.238138 (+ 571us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:04:48.238141 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:04:48.238259 (+ 118us) client_negotiation.cc:769] Sending connection context
0811 19:04:48.238375 (+ 116us) client_negotiation.cc:240] Negotiation successful
0811 19:04:48.238516 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33,"mutex_wait_us":5}
I20260811 19:04:48.238886 6436 log.cc:824] T 00000000000000000000000000000000 P 8113a711c6ff4a2897e089ca7dba484a: Log is configured to *not* fsync() on all Append() calls
I20260811 19:04:48.239367 6522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:48.229358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35098 (local address 127.13.109.60:39453)
0811 19:04:48.229766 (+ 408us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:48.229779 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:48.229818 (+ 39us) server_negotiation.cc:407] Connection header received
0811 19:04:48.229850 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:48.229855 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:48.229909 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:48.229988 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:48.230456 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:48.231714 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:48.232484 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:48.232744 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:48.232857 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:48.235180 (+ 2323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:04:48.235183 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:04:48.235186 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:04:48.235208 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:04:48.236797 (+ 1589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:48.237281 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:48.237287 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:48.237290 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:48.237378 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:48.237667 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:48.237670 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:48.237672 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:48.238006 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:04:48.238121 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:48.238458 (+ 337us) server_negotiation.cc:299] Negotiation successful
0811 19:04:48.238597 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":20,"server-negotiator.queue_time_us":341,"thread_start_us":133,"threads_started":1}
I20260811 19:04:48.239583 6442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:48.229920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32904 (local address 127.13.109.61:36705)
0811 19:04:48.230019 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:48.230023 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:48.230097 (+ 74us) server_negotiation.cc:407] Connection header received
0811 19:04:48.230227 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:48.230229 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:48.230270 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:48.230359 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:48.231077 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:48.231686 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:48.232700 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:48.232978 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:48.233080 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:48.236033 (+ 2953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:04:48.236038 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:04:48.236043 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:04:48.236075 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:04:48.237657 (+ 1582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:48.238218 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:48.238222 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:48.238224 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:48.238311 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:48.238674 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:48.238677 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:48.238679 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:48.238913 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:04:48.239107 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:48.239314 (+ 207us) server_negotiation.cc:299] Negotiation successful
0811 19:04:48.239437 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 19:04:48.241770 6436 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8113a711c6ff4a2897e089ca7dba484a: No bootstrap required, opened a new log
I20260811 19:04:48.242472 6517 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87e9f2c4ab994aac9d538c96bb1457d8: Bootstrap starting.
I20260811 19:04:48.243306 13748 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 19:04:48.243903 6357 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.109.60" port: 39453 } attempt: 2
I20260811 19:04:48.245656 6517 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 87e9f2c4ab994aac9d538c96bb1457d8: Neither blocks nor log segments found. Creating new log.
I20260811 19:04:48.245854 6436 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8113a711c6ff4a2897e089ca7dba484a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3cfcc2b5054946a3b031616c64ee1801" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 42667 } } peers { permanent_uuid: "8113a711c6ff4a2897e089ca7dba484a" member_type: VOTER last_known_addr { host: "127.13.109.61" port: 36705 } } peers { permanent_uuid: "87e9f2c4ab994aac9d538c96bb1457d8" member_type: VOTER last_known_addr { host: "127.13.109.60" port: 39453 } }
I20260811 19:04:48.244570 6331 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "3cfcc2b5054946a3b031616c64ee1801"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46684
I20260811 19:04:48.246112 6436 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8113a711c6ff4a2897e089ca7dba484a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8113a711c6ff4a2897e089ca7dba484a, State: Initialized, Role: FOLLOWER
I20260811 19:04:48.246482 6436 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8113a711c6ff4a2897e089ca7dba484a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3cfcc2b5054946a3b031616c64ee1801" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 42667 } } peers { permanent_uuid: "8113a711c6ff4a2897e089ca7dba484a" member_type: VOTER last_known_addr { host: "127.13.109.61" port: 36705 } } peers { permanent_uuid: "87e9f2c4ab994aac9d538c96bb1457d8" member_type: VOTER last_known_addr { host: "127.13.109.60" port: 39453 } }
I20260811 19:04:48.246665 6517 log.cc:824] T 00000000000000000000000000000000 P 87e9f2c4ab994aac9d538c96bb1457d8: Log is configured to *not* fsync() on all Append() calls
I20260811 19:04:48.247824 6524 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8113a711c6ff4a2897e089ca7dba484a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3cfcc2b5054946a3b031616c64ee1801" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 42667 } } peers { permanent_uuid: "8113a711c6ff4a2897e089ca7dba484a" member_type: VOTER last_known_addr { host: "127.13.109.61" port: 36705 } } peers { permanent_uuid: "87e9f2c4ab994aac9d538c96bb1457d8" member_type: VOTER last_known_addr { host: "127.13.109.60" port: 39453 } } }
I20260811 19:04:48.247977 6524 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8113a711c6ff4a2897e089ca7dba484a [sys.catalog]: This master's current role is: FOLLOWER
I20260811 19:04:48.248010 6436 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8113a711c6ff4a2897e089ca7dba484a [sys.catalog]: configured and running, proceeding with master startup.
Aug 11 19:04:48 dist-test-slave-0kls krb5kdc[6270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475087, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.109.62@KRBTEST.COM for kudu/127.13.109.60@KRBTEST.COM
I20260811 19:04:48.252794 6517 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87e9f2c4ab994aac9d538c96bb1457d8: No bootstrap required, opened a new log
W20260811 19:04:48.253940 6538 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8113a711c6ff4a2897e089ca7dba484a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 19:04:48.254088 6538 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 19:04:48.255777 6360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:48.244363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.109.60:39453 (local address 127.0.0.1:35110)
0811 19:04:48.244620 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0811 19:04:48.244636 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 19:04:48.244732 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:04:48.246558 (+ 1826us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:04:48.246563 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:04:48.246809 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:04:48.246930 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:48.246939 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:48.247737 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:48.247740 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:48.248480 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:04:48.248496 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:04:48.249129 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:04:48.249133 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:04:48.249223 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:48.249844 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:04:48.249867 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:04:48.251555 (+ 1688us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:04:48.253954 (+ 2399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:04:48.253960 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:04:48.253966 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:04:48.254248 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:04:48.254586 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:04:48.254589 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:04:48.254591 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:04:48.254702 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:04:48.255206 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:04:48.255209 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:04:48.255319 (+ 110us) client_negotiation.cc:769] Sending connection context
0811 19:04:48.255425 (+ 106us) client_negotiation.cc:240] Negotiation successful
0811 19:04:48.255585 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175}
I20260811 19:04:48.252198 6365 master_runner.cc:186] Error getting master registration for 127.13.109.62:42667: OK, instance_id { permanent_uuid: "3cfcc2b5054946a3b031616c64ee1801" instance_seqno: 1786475087707193 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 19:04:48.256230 6365 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 19:04:48.256440 6522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:48.244871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35110 (local address 127.13.109.60:39453)
0811 19:04:48.246285 (+ 1414us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:48.246290 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:48.246307 (+ 17us) server_negotiation.cc:407] Connection header received
0811 19:04:48.246350 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:48.246353 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:48.246400 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:48.246488 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:48.247047 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:48.247591 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:48.248625 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:48.248996 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:48.249118 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:48.251958 (+ 2840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:04:48.251962 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:04:48.251964 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:04:48.251992 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:04:48.253764 (+ 1772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:48.254383 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:48.254386 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:48.254388 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:48.254481 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:48.254812 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:48.254817 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:48.254819 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:48.255060 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:04:48.255156 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:48.256142 (+ 986us) server_negotiation.cc:299] Negotiation successful
0811 19:04:48.256281 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1282}
I20260811 19:04:48.257189 6517 raft_consensus.cc:348] T 00000000000000000000000000000000 P 87e9f2c4ab994aac9d538c96bb1457d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3cfcc2b5054946a3b031616c64ee1801" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 42667 } } peers { permanent_uuid: "8113a711c6ff4a2897e089ca7dba484a" member_type: VOTER last_known_addr { host: "127.13.109.61" port: 36705 } } peers { permanent_uuid: "87e9f2c4ab994aac9d538c96bb1457d8" member_type: VOTER last_known_addr { host: "127.13.109.60" port: 39453 } }
I20260811 19:04:48.257447 6517 raft_consensus.cc:729] T 00000000000000000000000000000000 P 87e9f2c4ab994aac9d538c96bb1457d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87e9f2c4ab994aac9d538c96bb1457d8, State: Initialized, Role: FOLLOWER
I20260811 19:04:48.257827 6517 consensus_queue.cc:260] T 00000000000000000000000000000000 P 87e9f2c4ab994aac9d538c96bb1457d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3cfcc2b5054946a3b031616c64ee1801" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 42667 } } peers { permanent_uuid: "8113a711c6ff4a2897e089ca7dba484a" member_type: VOTER last_known_addr { host: "127.13.109.61" port: 36705 } } peers { permanent_uuid: "87e9f2c4ab994aac9d538c96bb1457d8" member_type: VOTER last_known_addr { host: "127.13.109.60" port: 39453 } }
I20260811 19:04:48.259279 6517 sys_catalog.cc:564] T 00000000000000000000000000000000 P 87e9f2c4ab994aac9d538c96bb1457d8 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:04:48.259891 6357 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3cfcc2b5054946a3b031616c64ee1801: Bootstrap starting.
I20260811 19:04:48.262249 6540 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87e9f2c4ab994aac9d538c96bb1457d8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3cfcc2b5054946a3b031616c64ee1801" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 42667 } } peers { permanent_uuid: "8113a711c6ff4a2897e089ca7dba484a" member_type: VOTER last_known_addr { host: "127.13.109.61" port: 36705 } } peers { permanent_uuid: "87e9f2c4ab994aac9d538c96bb1457d8" member_type: VOTER last_known_addr { host: "127.13.109.60" port: 39453 } } }
I20260811 19:04:48.262365 6540 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87e9f2c4ab994aac9d538c96bb1457d8 [sys.catalog]: This master's current role is: FOLLOWER
W20260811 19:04:48.265264 6552 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 87e9f2c4ab994aac9d538c96bb1457d8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 19:04:48.265355 6552 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 19:04:48.263192 6443 master_runner.cc:186] Error getting master registration for 127.13.109.62:42667: OK, instance_id { permanent_uuid: "3cfcc2b5054946a3b031616c64ee1801" instance_seqno: 1786475087707193 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 19:04:48.266776 6443 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 19:04:48.268028 6357 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3cfcc2b5054946a3b031616c64ee1801: Neither blocks nor log segments found. Creating new log.
I20260811 19:04:48.269038 6357 log.cc:824] T 00000000000000000000000000000000 P 3cfcc2b5054946a3b031616c64ee1801: Log is configured to *not* fsync() on all Append() calls
I20260811 19:04:48.271972 6357 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3cfcc2b5054946a3b031616c64ee1801: No bootstrap required, opened a new log
I20260811 19:04:48.275347 6357 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3cfcc2b5054946a3b031616c64ee1801 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3cfcc2b5054946a3b031616c64ee1801" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 42667 } } peers { permanent_uuid: "8113a711c6ff4a2897e089ca7dba484a" member_type: VOTER last_known_addr { host: "127.13.109.61" port: 36705 } } peers { permanent_uuid: "87e9f2c4ab994aac9d538c96bb1457d8" member_type: VOTER last_known_addr { host: "127.13.109.60" port: 39453 } }
I20260811 19:04:48.275653 6357 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3cfcc2b5054946a3b031616c64ee1801 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3cfcc2b5054946a3b031616c64ee1801, State: Initialized, Role: FOLLOWER
I20260811 19:04:48.276033 6357 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3cfcc2b5054946a3b031616c64ee1801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3cfcc2b5054946a3b031616c64ee1801" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 42667 } } peers { permanent_uuid: "8113a711c6ff4a2897e089ca7dba484a" member_type: VOTER last_known_addr { host: "127.13.109.61" port: 36705 } } peers { permanent_uuid: "87e9f2c4ab994aac9d538c96bb1457d8" member_type: VOTER last_known_addr { host: "127.13.109.60" port: 39453 } }
I20260811 19:04:48.276743 6555 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3cfcc2b5054946a3b031616c64ee1801 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3cfcc2b5054946a3b031616c64ee1801" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 42667 } } peers { permanent_uuid: "8113a711c6ff4a2897e089ca7dba484a" member_type: VOTER last_known_addr { host: "127.13.109.61" port: 36705 } } peers { permanent_uuid: "87e9f2c4ab994aac9d538c96bb1457d8" member_type: VOTER last_known_addr { host: "127.13.109.60" port: 39453 } } }
I20260811 19:04:48.276983 6555 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3cfcc2b5054946a3b031616c64ee1801 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 19:04:48.277561 6357 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3cfcc2b5054946a3b031616c64ee1801 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:04:48.277613 6331 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3cfcc2b5054946a3b031616c64ee1801 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260811 19:04:48.277745 6331 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3cfcc2b5054946a3b031616c64ee1801 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:04:48.279208 6331 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3cfcc2b5054946a3b031616c64ee1801 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3cfcc2b5054946a3b031616c64ee1801" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 42667 } } peers { permanent_uuid: "8113a711c6ff4a2897e089ca7dba484a" member_type: VOTER last_known_addr { host: "127.13.109.61" port: 36705 } } peers { permanent_uuid: "87e9f2c4ab994aac9d538c96bb1457d8" member_type: VOTER last_known_addr { host: "127.13.109.60" port: 39453 } }
I20260811 19:04:48.280328 6331 leader_election.cc:288] T 00000000000000000000000000000000 P 3cfcc2b5054946a3b031616c64ee1801 [CANDIDATE]: Term 1 election: Requested vote from peers 8113a711c6ff4a2897e089ca7dba484a (127.13.109.61:36705), 87e9f2c4ab994aac9d538c96bb1457d8 (127.13.109.60:39453)
I20260811 19:04:48.281576 6489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3cfcc2b5054946a3b031616c64ee1801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "87e9f2c4ab994aac9d538c96bb1457d8"
I20260811 19:04:48.281903 6489 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 87e9f2c4ab994aac9d538c96bb1457d8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:04:48.283360 6489 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 87e9f2c4ab994aac9d538c96bb1457d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3cfcc2b5054946a3b031616c64ee1801 in term 1.
I20260811 19:04:48.283933 6409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3cfcc2b5054946a3b031616c64ee1801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "8113a711c6ff4a2897e089ca7dba484a"
I20260811 19:04:48.284204 6409 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8113a711c6ff4a2897e089ca7dba484a [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:04:48.285561 6409 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8113a711c6ff4a2897e089ca7dba484a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3cfcc2b5054946a3b031616c64ee1801 in term 1.
I20260811 19:04:48.283926 6303 leader_election.cc:302] T 00000000000000000000000000000000 P 3cfcc2b5054946a3b031616c64ee1801 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3cfcc2b5054946a3b031616c64ee1801, 87e9f2c4ab994aac9d538c96bb1457d8; no voters:
I20260811 19:04:48.287262 6555 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3cfcc2b5054946a3b031616c64ee1801 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:04:48.287850 6555 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3cfcc2b5054946a3b031616c64ee1801 [term 1 LEADER]: Becoming Leader. State: Replica: 3cfcc2b5054946a3b031616c64ee1801, State: Running, Role: LEADER
I20260811 19:04:48.288237 6555 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3cfcc2b5054946a3b031616c64ee1801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3cfcc2b5054946a3b031616c64ee1801" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 42667 } } peers { permanent_uuid: "8113a711c6ff4a2897e089ca7dba484a" member_type: VOTER last_known_addr { host: "127.13.109.61" port: 36705 } } peers { permanent_uuid: "87e9f2c4ab994aac9d538c96bb1457d8" member_type: VOTER last_known_addr { host: "127.13.109.60" port: 39453 } }
I20260811 19:04:48.291254 6572 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3cfcc2b5054946a3b031616c64ee1801 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3cfcc2b5054946a3b031616c64ee1801. Latest consensus state: current_term: 1 leader_uuid: "3cfcc2b5054946a3b031616c64ee1801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3cfcc2b5054946a3b031616c64ee1801" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 42667 } } peers { permanent_uuid: "8113a711c6ff4a2897e089ca7dba484a" member_type: VOTER last_known_addr { host: "127.13.109.61" port: 36705 } } peers { permanent_uuid: "87e9f2c4ab994aac9d538c96bb1457d8" member_type: VOTER last_known_addr { host: "127.13.109.60" port: 39453 } } }
I20260811 19:04:48.291386 6572 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3cfcc2b5054946a3b031616c64ee1801 [sys.catalog]: This master's current role is: LEADER
W20260811 19:04:48.293493 6567 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3cfcc2b5054946a3b031616c64ee1801: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 19:04:48.293582 6567 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 19:04:48.296814 6574 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:04:48.297894 6574 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:04:48.304662 6489 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 87e9f2c4ab994aac9d538c96bb1457d8 [term 1 FOLLOWER]: Refusing update from remote peer 3cfcc2b5054946a3b031616c64ee1801: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:04:48.305856 6555 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3cfcc2b5054946a3b031616c64ee1801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "87e9f2c4ab994aac9d538c96bb1457d8" member_type: VOTER last_known_addr { host: "127.13.109.60" port: 39453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:04:48.311578 6540 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87e9f2c4ab994aac9d538c96bb1457d8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3cfcc2b5054946a3b031616c64ee1801. Latest consensus state: current_term: 1 leader_uuid: "3cfcc2b5054946a3b031616c64ee1801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3cfcc2b5054946a3b031616c64ee1801" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 42667 } } peers { permanent_uuid: "8113a711c6ff4a2897e089ca7dba484a" member_type: VOTER last_known_addr { host: "127.13.109.61" port: 36705 } } peers { permanent_uuid: "87e9f2c4ab994aac9d538c96bb1457d8" member_type: VOTER last_known_addr { host: "127.13.109.60" port: 39453 } } }
I20260811 19:04:48.311753 6540 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87e9f2c4ab994aac9d538c96bb1457d8 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 19:04:48.313524 6409 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8113a711c6ff4a2897e089ca7dba484a [term 1 FOLLOWER]: Refusing update from remote peer 3cfcc2b5054946a3b031616c64ee1801: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:04:48.314342 6555 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3cfcc2b5054946a3b031616c64ee1801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8113a711c6ff4a2897e089ca7dba484a" member_type: VOTER last_known_addr { host: "127.13.109.61" port: 36705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:04:48.318158 6572 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3cfcc2b5054946a3b031616c64ee1801 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3cfcc2b5054946a3b031616c64ee1801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3cfcc2b5054946a3b031616c64ee1801" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 42667 } } peers { permanent_uuid: "8113a711c6ff4a2897e089ca7dba484a" member_type: VOTER last_known_addr { host: "127.13.109.61" port: 36705 } } peers { permanent_uuid: "87e9f2c4ab994aac9d538c96bb1457d8" member_type: VOTER last_known_addr { host: "127.13.109.60" port: 39453 } } }
I20260811 19:04:48.318329 6572 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3cfcc2b5054946a3b031616c64ee1801 [sys.catalog]: This master's current role is: LEADER
I20260811 19:04:48.318775 6540 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87e9f2c4ab994aac9d538c96bb1457d8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3cfcc2b5054946a3b031616c64ee1801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3cfcc2b5054946a3b031616c64ee1801" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 42667 } } peers { permanent_uuid: "8113a711c6ff4a2897e089ca7dba484a" member_type: VOTER last_known_addr { host: "127.13.109.61" port: 36705 } } peers { permanent_uuid: "87e9f2c4ab994aac9d538c96bb1457d8" member_type: VOTER last_known_addr { host: "127.13.109.60" port: 39453 } } }
I20260811 19:04:48.318908 6540 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87e9f2c4ab994aac9d538c96bb1457d8 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 19:04:48.319429 6580 mvcc.cc:205] Tried to move back new op lower bound from 7317401961685917696 to 7317401961632460800. Current Snapshot: MvccSnapshot[applied={T|T < 7317401961685917696}]
I20260811 19:04:48.322167 6524 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8113a711c6ff4a2897e089ca7dba484a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3cfcc2b5054946a3b031616c64ee1801. Latest consensus state: current_term: 1 leader_uuid: "3cfcc2b5054946a3b031616c64ee1801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3cfcc2b5054946a3b031616c64ee1801" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 42667 } } peers { permanent_uuid: "8113a711c6ff4a2897e089ca7dba484a" member_type: VOTER last_known_addr { host: "127.13.109.61" port: 36705 } } peers { permanent_uuid: "87e9f2c4ab994aac9d538c96bb1457d8" member_type: VOTER last_known_addr { host: "127.13.109.60" port: 39453 } } }
I20260811 19:04:48.322337 6524 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8113a711c6ff4a2897e089ca7dba484a [sys.catalog]: This master's current role is: FOLLOWER
I20260811 19:04:48.323324 6579 mvcc.cc:205] Tried to move back new op lower bound from 7317401961685917696 to 7317401961632460800. Current Snapshot: MvccSnapshot[applied={T|T < 7317401961685917696}]
I20260811 19:04:48.334328 6361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:48.299147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46696 (local address 127.13.109.62:42667)
0811 19:04:48.299259 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:48.299264 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:48.308189 (+ 8925us) server_negotiation.cc:407] Connection header received
0811 19:04:48.310541 (+ 2352us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:48.310546 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:48.310604 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:48.310672 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:48.311751 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:48.312529 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:48.322695 (+ 10166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:48.323011 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:48.323121 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:48.328898 (+ 5777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:04:48.328903 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:04:48.328906 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:04:48.328940 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:04:48.330700 (+ 1760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:48.331602 (+ 902us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:48.331607 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:48.331609 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:48.331721 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:48.332465 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:48.332469 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:48.332470 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:48.332743 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:04:48.332856 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:48.333998 (+ 1142us) server_negotiation.cc:299] Negotiation successful
0811 19:04:48.334153 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 19:04:48.335289 6442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:48.297751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32908 (local address 127.13.109.61:36705)
0811 19:04:48.297923 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:48.297929 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:48.306437 (+ 8508us) server_negotiation.cc:407] Connection header received
0811 19:04:48.306697 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:48.306702 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:48.306763 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:48.306819 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:48.308278 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:48.309092 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:48.321173 (+ 12081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:48.321573 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:48.321711 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:48.326894 (+ 5183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:04:48.326899 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:04:48.326902 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:04:48.326935 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:04:48.328702 (+ 1767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:48.332044 (+ 3342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:48.332048 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:48.332050 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:48.332152 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:48.333187 (+ 1035us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:48.333191 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:48.333193 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:48.333448 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:04:48.333588 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:48.334978 (+ 1390us) server_negotiation.cc:299] Negotiation successful
0811 19:04:48.335129 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260811 19:04:48.344489 6522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:48.298298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35122 (local address 127.13.109.60:39453)
0811 19:04:48.298417 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:48.298422 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:48.321563 (+ 23141us) server_negotiation.cc:407] Connection header received
0811 19:04:48.321711 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:48.321715 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:48.321776 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:48.321886 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:48.334152 (+ 12266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:48.334870 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:48.338296 (+ 3426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:48.338646 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:48.338736 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:48.340694 (+ 1958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:04:48.340698 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:04:48.340701 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:04:48.340730 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:04:48.342478 (+ 1748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:48.343002 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:48.343006 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:48.343008 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:48.343103 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:04:48.343427 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:04:48.343430 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:04:48.343432 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:04:48.343669 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:04:48.343807 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:48.344206 (+ 399us) server_negotiation.cc:299] Negotiation successful
0811 19:04:48.344333 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 19:04:48.345234 6555 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3cfcc2b5054946a3b031616c64ee1801 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3cfcc2b5054946a3b031616c64ee1801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3cfcc2b5054946a3b031616c64ee1801" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 42667 } } peers { permanent_uuid: "8113a711c6ff4a2897e089ca7dba484a" member_type: VOTER last_known_addr { host: "127.13.109.61" port: 36705 } } peers { permanent_uuid: "87e9f2c4ab994aac9d538c96bb1457d8" member_type: VOTER last_known_addr { host: "127.13.109.60" port: 39453 } } }
I20260811 19:04:48.345417 6555 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3cfcc2b5054946a3b031616c64ee1801 [sys.catalog]: This master's current role is: LEADER
I20260811 19:04:48.346788 6524 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8113a711c6ff4a2897e089ca7dba484a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3cfcc2b5054946a3b031616c64ee1801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3cfcc2b5054946a3b031616c64ee1801" member_type: VOTER last_known_addr { host: "127.13.109.62" port: 42667 } } peers { permanent_uuid: "8113a711c6ff4a2897e089ca7dba484a" member_type: VOTER last_known_addr { host: "127.13.109.61" port: 36705 } } peers { permanent_uuid: "87e9f2c4ab994aac9d538c96bb1457d8" member_type: VOTER last_known_addr { host: "127.13.109.60" port: 39453 } } }
I20260811 19:04:48.346932 6524 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8113a711c6ff4a2897e089ca7dba484a [sys.catalog]: This master's current role is: FOLLOWER
I20260811 19:04:48.348275 6574 catalog_manager.cc:1347] Generated new cluster ID: 1c8a9cba5a7c46d6bd636d95fcd6c49d
I20260811 19:04:48.348346 6574 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:04:48.358314 6584 mvcc.cc:205] Tried to move back new op lower bound from 7317401961685917696 to 7317401961632460800. Current Snapshot: MvccSnapshot[applied={T|T < 7317401961685917696}]
I20260811 19:04:48.377185 6574 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:04:48.378695 6574 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:04:48.395920 6574 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3cfcc2b5054946a3b031616c64ee1801: Generated new TSK 0
I20260811 19:04:48.396974 6574 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 19:04:48.434312 6442 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:04:48.431010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32910 (local address 127.13.109.61:36705)
0811 19:04:48.431175 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:48.431180 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:48.432762 (+ 1582us) server_negotiation.cc:407] Connection header received
0811 19:04:48.433289 (+ 527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:48.433294 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:48.433352 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:48.433408 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:48.434158 (+ 750us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:32910: BlockingRecv error: recv got EOF from 127.0.0.1:32910 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
W20260811 19:04:48.435650 6361 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:04:48.431723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46708 (local address 127.13.109.62:42667)
0811 19:04:48.431844 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:48.431848 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:48.432555 (+ 707us) server_negotiation.cc:407] Connection header received
0811 19:04:48.434776 (+ 2221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:48.434780 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:48.434847 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:48.434904 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:48.435516 (+ 612us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46708: BlockingRecv error: recv got EOF from 127.0.0.1:46708 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260811 19:04:48.436983 6522 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:04:48.431233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35124 (local address 127.13.109.60:39453)
0811 19:04:48.431355 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:48.431360 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:48.433194 (+ 1834us) server_negotiation.cc:407] Connection header received
0811 19:04:48.436195 (+ 3001us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:48.436199 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:48.436254 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:48.436307 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:48.436854 (+ 547us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35124: BlockingRecv error: recv got EOF from 127.0.0.1:35124 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260811 19:04:48.442685 6442 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:04:48.440766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32914 (local address 127.13.109.61:36705)
0811 19:04:48.440893 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:48.440897 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:48.441495 (+ 598us) server_negotiation.cc:407] Connection header received
0811 19:04:48.441742 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:48.441746 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:48.441901 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:48.441961 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:48.442564 (+ 603us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:32914: BlockingRecv error: recv got EOF from 127.0.0.1:32914 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260811 19:04:48.448850 6442 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:04:48.446997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32928 (local address 127.13.109.61:36705)
0811 19:04:48.447136 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:48.447141 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:48.447743 (+ 602us) server_negotiation.cc:407] Connection header received
0811 19:04:48.447979 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:48.447985 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:48.448048 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:48.448106 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:48.448715 (+ 609us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:32928: BlockingRecv error: recv got EOF from 127.0.0.1:32928 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260811 19:04:48.455374 6442 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:04:48.453587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32936 (local address 127.13.109.61:36705)
0811 19:04:48.453710 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:48.453715 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:48.454320 (+ 605us) server_negotiation.cc:407] Connection header received
0811 19:04:48.454520 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:48.454523 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:48.454586 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:48.454644 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:48.455247 (+ 603us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:32936: BlockingRecv error: recv got EOF from 127.0.0.1:32936 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260811 19:04:48.464491 6442 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:04:48.462402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32940 (local address 127.13.109.61:36705)
0811 19:04:48.462572 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:48.462577 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:48.463275 (+ 698us) server_negotiation.cc:407] Connection header received
0811 19:04:48.463521 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:48.463525 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:48.463587 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:48.463644 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:48.464358 (+ 714us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:32940: BlockingRecv error: recv got EOF from 127.0.0.1:32940 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
W20260811 19:04:48.477708 6442 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:04:48.475672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32946 (local address 127.13.109.61:36705)
0811 19:04:48.475812 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:48.475817 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:48.476514 (+ 697us) server_negotiation.cc:407] Connection header received
0811 19:04:48.476724 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:48.476727 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:48.476793 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:48.476899 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:48.477574 (+ 675us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:32946: BlockingRecv error: recv got EOF from 127.0.0.1:32946 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260811 19:04:48.497706 6442 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:04:48.496364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32958 (local address 127.13.109.61:36705)
0811 19:04:48.496503 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:48.496508 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:48.496707 (+ 199us) server_negotiation.cc:407] Connection header received
0811 19:04:48.496861 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:48.496865 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:48.496927 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:48.497037 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:48.497573 (+ 536us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:32958: BlockingRecv error: recv got EOF from 127.0.0.1:32958 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260811 19:04:48.533478 6442 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:04:48.532210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32966 (local address 127.13.109.61:36705)
0811 19:04:48.532391 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:48.532396 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:48.532520 (+ 124us) server_negotiation.cc:407] Connection header received
0811 19:04:48.532712 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:48.532715 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:48.532777 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:48.532874 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:48.533354 (+ 480us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:32966: BlockingRecv error: recv got EOF from 127.0.0.1:32966 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
W20260811 19:04:48.601100 6442 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:04:48.599977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32976 (local address 127.13.109.61:36705)
0811 19:04:48.600128 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:48.600132 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:48.600236 (+ 104us) server_negotiation.cc:407] Connection header received
0811 19:04:48.600347 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:48.600349 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:48.600401 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:48.600484 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:48.600988 (+ 504us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:32976: BlockingRecv error: recv got EOF from 127.0.0.1:32976 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260811 19:04:48.732820 6442 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:04:48.731528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32990 (local address 127.13.109.61:36705)
0811 19:04:48.731718 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:48.731721 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:48.731852 (+ 131us) server_negotiation.cc:407] Connection header received
0811 19:04:48.731943 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:48.731946 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:48.732024 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:48.732154 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:48.732711 (+ 557us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:32990: BlockingRecv error: recv got EOF from 127.0.0.1:32990 (error 108)
Metrics: {"server-negotiator.queue_time_us":98}
W20260811 19:04:48.992641 6442 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:04:48.991538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33004 (local address 127.13.109.61:36705)
0811 19:04:48.991706 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:48.991710 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:48.991726 (+ 16us) server_negotiation.cc:407] Connection header received
0811 19:04:48.991870 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:48.991872 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:48.991923 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:48.992002 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:04:48.992507 (+ 505us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33004: BlockingRecv error: recv got EOF from 127.0.0.1:33004 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 19:04:49.255553 6538 catalog_manager.cc:1259] Loaded cluster ID: 1c8a9cba5a7c46d6bd636d95fcd6c49d
I20260811 19:04:49.255635 6538 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8113a711c6ff4a2897e089ca7dba484a: loading cluster ID for follower catalog manager: success
I20260811 19:04:49.257361 6538 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8113a711c6ff4a2897e089ca7dba484a: acquiring CA information for follower catalog manager: success
I20260811 19:04:49.258586 6538 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8113a711c6ff4a2897e089ca7dba484a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 19:04:49.266809 6552 catalog_manager.cc:1259] Loaded cluster ID: 1c8a9cba5a7c46d6bd636d95fcd6c49d
I20260811 19:04:49.266898 6552 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 87e9f2c4ab994aac9d538c96bb1457d8: loading cluster ID for follower catalog manager: success
I20260811 19:04:49.268559 6552 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 87e9f2c4ab994aac9d538c96bb1457d8: acquiring CA information for follower catalog manager: success
I20260811 19:04:49.269841 6552 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 87e9f2c4ab994aac9d538c96bb1457d8: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 19:04:49.511325 6657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:49.507517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33020 (local address 127.13.109.61:36705)
0811 19:04:49.507768 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:49.507772 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:49.507787 (+ 15us) server_negotiation.cc:407] Connection header received
0811 19:04:49.507841 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:49.507844 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:49.507976 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:49.508093 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:04:49.508330 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:49.508898 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:49.509725 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:49.509967 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:49.510045 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:49.510441 (+ 396us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:04:49.510919 (+ 478us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:04:49.511013 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:49.511138 (+ 125us) server_negotiation.cc:299] Negotiation successful
0811 19:04:49.511195 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":94,"threads_started":1}
I20260811 19:04:49.518191 6664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:49.514602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35140 (local address 127.13.109.60:39453)
0811 19:04:49.514855 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:49.514859 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:49.514872 (+ 13us) server_negotiation.cc:407] Connection header received
0811 19:04:49.514953 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:49.514956 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:49.515068 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:49.515184 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:04:49.515375 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:49.515883 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:49.516659 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:49.516873 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:49.516949 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:49.517224 (+ 275us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:04:49.517698 (+ 474us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:04:49.517842 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:49.517934 (+ 92us) server_negotiation.cc:299] Negotiation successful
0811 19:04:49.518002 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":77,"threads_started":1}
I20260811 19:04:49.525774 6657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:49.521172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33030 (local address 127.13.109.61:36705)
0811 19:04:49.521316 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:49.521323 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:49.521560 (+ 237us) server_negotiation.cc:407] Connection header received
0811 19:04:49.521800 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:49.521807 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:49.521869 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:49.521919 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:04:49.522353 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:49.522889 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:49.524229 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:49.524570 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:49.524651 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:49.525246 (+ 595us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:04:49.525315 (+ 69us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:04:49.525399 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:49.525522 (+ 123us) server_negotiation.cc:299] Negotiation successful
0811 19:04:49.525586 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 19:04:49.525900 6672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:49.521517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46724 (local address 127.13.109.62:42667)
0811 19:04:49.521768 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:49.521774 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:49.521814 (+ 40us) server_negotiation.cc:407] Connection header received
0811 19:04:49.521912 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:49.521915 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:49.522024 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:49.522147 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:04:49.522437 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:49.523240 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:49.524192 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:49.524457 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:49.524578 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:49.524897 (+ 319us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:04:49.525203 (+ 306us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:04:49.525292 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:49.525641 (+ 349us) server_negotiation.cc:299] Negotiation successful
0811 19:04:49.525713 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":78,"threads_started":1}
I20260811 19:04:49.526006 6664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:04:49.521274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35142 (local address 127.13.109.60:39453)
0811 19:04:49.521405 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0811 19:04:49.521409 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:04:49.521645 (+ 236us) server_negotiation.cc:407] Connection header received
0811 19:04:49.521842 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:04:49.521845 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:04:49.521896 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:04:49.521943 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:04:49.522332 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:49.523109 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:49.524247 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:04:49.524589 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:04:49.524658 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:04:49.525226 (+ 568us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:04:49.525308 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:04:49.525384 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 19:04:49.525836 (+ 452us) server_negotiation.cc:299] Negotiation successful
0811 19:04:49.525887 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 19:04:49.536274 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 6284
I20260811 19:04:49.545674 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 6365
I20260811 19:04:49.555014 13748 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrGEMHD/build/debug/bin/kudu with pid 6443
2026-08-11T19:04:49Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3330 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6552 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (297051 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST